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INTRODUCTION 


The  present  report  summarizes  research  for  the  Defense 
Advanced  Research  Projects  Agency  to  design  an  advanced  system  for 
the  management  of  indications  and  warning  (I&W)  analysis.  That 
system,  designated  the  indications  and  warning  analysis  management 
system  (IWAMS),  is  computer-based  and  designed  to  perform  these 
functions : 

•  Monitor  selected  analytic  operations  at  DoD  I&W  centers 

•  Extract  and  display  measures  of  analytic  effectiveness, 
readiness  and  maintenance 

•  Facilitate  warning  analysis 

•  Be  compatible  with  normal  analytic  operations 

•  Be  capable  of  being  integrated  into  existing  computer- 
based  analyst  support  systems 

The  IWAMS  project  began  m  late  197S.  Members  of  the 
project  team  have  spent  approximately  2  months  on-site  at  i&W  centers 
observing  operations,  conducting  a  dialog  with  analysts  and  managers, 
and  testing  IWAMS  concepts.  We  have  built  a  model  of  the  process  of 
warning  analysis;  defined  analysis  performance  measures;  designed  a 
system  of  analysis  methods,  including  projection  techniques;  and 
designed  the  software  architecture  for  tne  IWAMS  computer-based 
component . 


Figure  1  is  an  artist's  simplified  graphic  representation 
of  the  IWAMS  concept.  The  largest  rectangle  symbolically  reflects  two 
tasks  completed  on  the  project:  modeling  the  process  of  warning 
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analysis  and  the  design  of  new  analytic  procedures.  The  model  is 
composed  of  three  major  stages:  monitoring,  threat  recognition  and 
projection,  each  involving  numerous  procedures  and  steps.  The 
second  task  culminated  in  the  design  of  analytic  methods  to  be  used 
in  each  of  these  three  major  stages.  IWAMS  is  intended  to  provide 
selective  computer  support  to  the  analyst  in  each  stage. 

The  detail  in  the  other  two  rectangles  conveys  the  concept 
of  computer-based  warning  analysis  management.  IWAMS  is  designed  to 
provide  managers  access  to  the  status  of  analysis  and  to  enhance 
management  feedback.  The  circle  in  the  top  right  rectangle  under¬ 
scores  the  essential  need  for  measures  of  analytic  effectiveness, 
readiness,  and  maintenance.  The  development  of  these  measures  was  a 
major  and  challenging  task. 

Overall,  Figure  1  correctly  depicts  IWAMS  as  a  computer- 
based  cybernetic  system  in  which  man-machine  interactions  which  occur 
during  the  pursuit  of  warning  analysis  generate  data  for  measuring 
analysis  performance  and  fostering  productive  interaction  among 
analysts  and  managers.  The  system  is  described  in  detail  in  the 
body  of  the  report. 


IWAMS  is  an  advanced  system  concept  The  designers  have 
spent  substantial  time  on-site  at  warning  centers  and  have  become 
familiar  with  warning  analysis  operations  and  the  support  systems 
for  analysts.  Wo  have  discussed  fundamental  issues  of  warning  with 
managers  and  analysts  now  or  formerly  in  the  warning  community.  We 
have  reviewed  most  of  the  basic  documentation  on  the  warning  problem 
recently  generated  by  the  Department  of  Defense,  other  government 
agencies,  and  congressional  committees.  We  are  knowledgeable  in  the 
general  literature  of  warning  and  related  intelligence  problems. 

Our  purpose,  however,  is  not  to  review  or  comment  at  length  on 
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current  warning  operations.  Our  purpose,  difficult  enough,  is  to 
develop  practical  and  useful  advanced  concepts  for  the  management 
and  performance  of  warning  analysis,  particularly  with  reference  to 
computer-based  technology  and  its  supporting  role  in  human  analytic 
operations.  We  have  not  been  directly  concerned  with  the  problems  of 
data  collection. 

Part  of  the  difficulty  derives  from  long-standing  political, 
institutional  and  cultural  obstacles  to  improved  warning  by  the  U.S. 
Documented  more  fully  of  late,  these  obstacles  are  indeed  formidable. 
However,  advanced  research  has  great  potential  for  surmounting  these 
obstacles  and  hence  a  strong  obligation  to  contribute  substantially 
to  the  improvement  of  warning  capability.  It  is  not  difficult  to  see 
why  this  improvement  is  needed.  Warning  is  central  to  the  intelli¬ 
gence  process  and  effective  intelligence  is  central  to  national 
security  in  an  increasingly  dangerous  world  where  the  premium  on 
alertness  is  rapidly  rising. 

There  is  also  another,  very  fundamental  constraint: 
conceptual  obstacles,  problems  of  method,  which  are  not  overcome  by 
decree  and  according  to  schedules.  As  one  thinks  increasingly  about 
the  warning  issue  and  explores  both  the  history  and  current  opera¬ 
tions,  the  tendency  is  strongly  to  believe  that  perhaps  among  the 
greatest  challenges  are  a  series  of  conceptual  problems  at  the  heart 
of  analysis  and  interpretation.  As  the  present  research  was  intended 
tc  establish,  some  of  these  problems  are  known  to  us,  can  be  under¬ 
stood  and,  given  our  technology,  appear  subject  to  amelioration.  But 
clearly  we  must  increase  our  understanding  at  the  conceptual  level  in 
order  to  apply  one  of  our  great  weapons,  advanced  technology  —  in  this 
case,  primarily  information  technology  —  successfully  to  the  problem. 

The  watershed  literature  of  warning  exhibits  both  diagnos¬ 
tic  and  prescriptive  tendencies.  Post  mortems  such  as  Wohlstetter ' s 
Pearl  Harbor,  Whaley’s  Codeword  Barbarossa,  and  Schlaim's  National 
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Intelligence  Failures;  The  Case  of  the  Yom  Kipper  War  are  classics 
with,  in  our  view,  primarily  a  diagnostic  thrust.  Although  not 
focused  strongly  on  the  analysis  problem  as  such,  the  historical 
lessons  laid  out  are  highly  relevant.  The  prescriptive  aspect  is 
represented  by  a  set  of  recent  writings  by  Thomas  Relden,  members 
of  the  Defense  Intelligence  Agency  (DIA)  and  various  others  that  is 
focused  primarily,  though  certainly  not  exclusively,  on  present 
warning  operations,  particularly  the  concept  of  a  network  or  community 
of  analysts.  What  we  hope  to  do  in  the  present  research  is  contribute 
to  both  diagnostic  and  prescriptive  thought  that  can  affect  the  future 
of  warning  analysis. 


Developed  at  length  in  the  body  of  the  text  is  a  fundamental 
and  complex  argument  behind  the  IWAMS  concept  that  we  should  summarize 
at  the  beginning, 

1.  At  present  there  is  an  epistemological  crisis  which 
throws  into  doubt  our  interpretive  ability,  our  success  in  making 
sense  of  events,  our  ability  to  project  outcomes.  There  is  a  deep 
pessimism  on  the  part  of  many  well-informed,  well-intentioned  people 
about  prospects  for  improving  human  capabilities  to  project  ahead  and 
anticipate  crises.  Theories,  both  formal  and  informal,  assert  that 
the  root  causes  of  our  difficulties  are  certain  very  old  psycnophysio- 
logical  constraints  on  the  human  mind.  Moreover,  these  root  causes 
are  percieved  as  being  continually  and  increasingly  exacerbated  by  the 
modern  information  deluge  (perceived  now  as  an  "overload"),  a  sort  of 
huge  and  peculiar  collage  added-to  daily  but  perversly  discontinuous 
which,  while  it  promises  much,  r.ow  frustrates  our  interpretive  impulse 
and  conditions  us  to  limited  perspectives.  It  tends  to  cut  off  our 
inquiry  into  these  problems.  There  is  a  siren  quality  about  it,  a 
kind  of  anesthetic  effect. 
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2.  Two  major  constraints  on  the  mind,  described 
variously,  are  considered  very  important  to  the  warning  problem. 

The  first  concerns  man's  difficulty  in  reconstructing  his  past 
analytic  perspectives.  In  greatly  simplified  terms,  it  is  a 
"failure"  of  memory  which  seems  naturally  to  occur  as  we  perform 
cognition  through  time;  we  forget  as  we  learn.  Discussed  in  detail 
below,  this  problem  impairs  post  mortems ,  thereby  curtailing  diag¬ 
nosis  and  prescription.  In  particular,  Richard  J.  Heuer,  Jr. .  has 
written  extremely  valuably  about  this  problem. 

The  second  constraint  is  the  difficulty  of  the  mind  in 
rapidly  perceiving  ramifications  in  meaning  through  complex  frame¬ 
works  of  coherent  perspectives  —  through,  in  short,  many  assumptions 
and  elements  of  logic  —  that  occur  when  individual  judgments  are 
made  about  small  portions  of  the  overall  framework.  In  essence  we 
have  difficulty  in  rapidly  realizing  implications .  However,  the 
vital  creation  of  complex  frames  of  reference,  arising  from  man's 
essential  strategy  of  building  models  of  past,  present,  and  future 
reality,  has  not  been  powerfully  accomplished  in  warning  operations 
owing  to  many  factors,  not  the  least  of  which  is  the  modern  informa¬ 
tion  overload.  Yet  we  desperately  need  higher  context  for  developing 
meaning  in  warning  analysis  and  to  mitigate  the  information  overload; 
and  we  intend  both  to  exemplify  such  contexts  and  to  show  that  the 
machine  will  help  in  providing  and  sustaining  them.  But  as  we 
develop  such  context  we  must  insure  that  the  human  analyst  can  see 
ramifications  of  meaning  throughout  it;  in  short,  that  the  analyst 
perceives  registrations  of  new  meaning. 

3 . ;  These  and  other  fundamental  obstacles  have  led  us  to 
solutions  that  involve  computer  technology  —  the  machine.  But  we 
must  be  emphatic  that  human  intelligence,  not  "artificial  intelli¬ 
gence,"  is  what,  we  seek  to  promote.  We  can  suggest  our  outlook  by 
citing  this  passage  by  Edward  Roths tein: 
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The  brain  itself  may  be  more  of  a  probabalistic  system 
than  a  formal  one,  more  of  a  cloud  than  a  clock,  to  use  Karl 
Popper's  metaphors,  and  its  interaction  with  the  rest  of  the  body 
would  also  make  it  difficult  to  consider  it  as  a  different  inde¬ 
pendent  system.  Researchers  in  AI  hope  they  will  never  have  to 
model  the  physiology  of  the  brain,-  they  concentrate  upon  the 
workings  of  the  mind,  attempting  to  mirror  its  features.  But 
this  project  seems  just  as  enormous.  The  achievement  of  AI's 
project  would  mean  that  one  would  have  a  program,  a  finite  set 
of  instructions,  that  would  give  a  literal  structure  of  the 
human  mind,  a  'string*  of  statements  in  which  one  could  read  a 
universal  grammar  of  creativity.  Such  a  project  is  probably  of 
the  same  order  of  difficulty  as  the  search  for  the  secrets  of 
life  itself. 

But  in  some  sense  this  dream  of  AI  is  the  archetypal 
dream  of  much  of  contemporary  inquiry.  It  is  not  to  find  a 
'universal  language'  whose  syntax  would  reveal  the  truth  of  the 
world  —  Godel  proved  such  a  language  to  be  impossible  --  but 
to  find  a  'universal  hermeneutics'  that  would  reduce  the 
fullness  of  the  world  to  an  underlying  syntax,  to  basic 
'structures',  and  that  would,  conversely,  be  able  to  read  the 
complexity  surrounding  us  in  these  basic  strings.;  This  is  the 
dream  not  only  of  AI  but  of  much  contemporary  genetics,  linguis¬ 
tics  ,  and  advanced  literary  theory  .... 

We  have  not  minimized  the  obstacles  to  effective  warning  analysis 
by  the  human  —  there  is  ample  discussion  of  these  below  —  but  it 
seems  obvious  to  us  on  every  level  that  no  machine  can  be  programmed 
to  substitute  for  the  human  analyst  in  any  final  sense.  Moreover, 
Godel' s  Incompleteness  Theorem  not  only  undermines  confidence  in  the 
"dream"  of  AI,  but  is  one  of  many  signs  that  reality  will  likely 
continue  to  surpass  our  models  and  theories. 


The  human  analyst  is  limited;  the  machine,  if  differently, 
is  much  more  limited;  and  we  find  ourselves  without  choice  engaged  in 
an  heuristic  process.  The  important  immediate  question  becomes:  how 
should  we  aid  the  now  largely  unaided  human  mind  in  performing  warning 
analysis?  To  sum  up  very  simply  the  major  thrust  of  our  research:  we 
argue  for  a  computer-based  extrasomatic  memory  of  analytic  operations 
which  in  its  various  forms  also  holds  the  analytic  context  developed 
heuristically  by  the  analysts,  themselves,  over  time.  The  machine 
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allows  us  -co  record  and  thereby  reconstruct  past  analytic  perspec¬ 
tives,  safely  removed  from  the  losses  of  human  memory.  At  the  same 
time,  the  machine  allows  us  —  indeed,  must  prompt  us  —  to  build  and 
preserve  as  well  as  review,  criticize,  dismantle  and  recreate,  the 
vital  context  for  making  warning  judgments.  Extrasomatic  is  an  appro¬ 
priate  term  because  it  connotes  something  apart  from,  but  also  an 
extension  of,  human  capabilities.  And  indeed,  for  many  reasons  the 
warning  analytic  context  we  speak  cf  as  being  stored  extrasomatically 
must  complement  the  schemata  (or  templates)  of  the  mind,  those  mental 
constructs  which  allow  us  to  recognize  portions  of  reality,  interpolate 
and  extrapolate,  infer,  and  perform  other  mental  operations.  We  give 
many  examples  below,  but  note  for  the  present  that  the  warning  analytic 
context  can  range  from  simple  chronological  models  of  hypothetical 
threat  situations  to  sophisticated  forecasting  maps.  But  what  we  are 
talking  about  here  are  forms  within  which  context  and  meaning  may 
change . 

Computer  technology  is  essential  for  a  number  of  reasons, 
many  of  which  can  be  summed  up  in  the  need  to  maintain  requisite 
speed  of  operations  to  complement  the  dynamics  of  human  cognitive 
processes.  For  this  reason  the  electronic  visual  medium  --  computer- 
driven  displays  and  their  rhetoric  such  as  color  coding  and  dynamic 
lines  —  becomes  increasingly  important.  By  these  means,  the  machine 
can  truly  extend  the  powers  of  the  human  mind  performing  warning 
analysis . 


The  incentive  for  such  analysis  basically  resides  in  our 
interpretive  impulse,  the  need  for  knowledge  for  survival.  The 
incentive  is  enhanced  by  virtue  of  the  extended  abilities,  and  hence 
satisfaction,  of  meeting  the  challenge  more  effectively.  This 
increase  in  incentive  should  apply  both  to  the  individual  analyst 
and  to  communities  of  analysts.  Indeed,  it  should  strengthen  both 
in  all  renpects. 
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4.  As  we  have  said,  the  development  of  more  effective 
methods  and  techniques  of  warning  analysis,  especially  projections, 
must  be  an  heuristic  process.  There  is  now  no  satisfactory  means  of 
projection  for  problems  in  international  warning.  But  the  inter¬ 
esting  issue  is  not  that  there  are  now  enormous  difficulties;  rather, 
it  is  in  the  prospects  for  improvement.  In  our  judgment  it  is 
arrogant  and  naive  to  suppose  that  prospects  are  meager.  We  have 
not  adequately  employed  man-machine  approaches  to  this  problem,  and 
we  will  not  mitigate  the  fundamental  constraints  on  the  unaided 
human  mind  in  doing  such  analysis  until  we  bring  computer  technology 
to  bear  more  powerfully,  and  in  the  hands  of  the  working  analyst. 

He  must  emerge  as  the  chief  innovator.  In  our  view,  the  speed  and 
breadth  of  analytic  operations  potentially  available  through  such 
computer-based  support  systems,  together  with  the  secure  memory 
these  systems  afford,  is  directly  related  to  enormous  possibilities 
of  increasingly  sophisticated  and  realistic  human  interpretation 
of  reality.  There  seems  a  clear  imperative  in  the  fact  that  very  few 
manhours  have  ever  been  expended  doing  such  analysis  in  actual  warning 
operations.  We  believe  we  have  developed  in  the  present  research  a 
promising  first-generation  approach,  a  beginning  that  can  be  refined 
by  analysts  doing  analysis. 

5.  Of  the  many  challenges  to  improving  the  effectiveness 
of  warning  analysis,  three  seem  especially  prominent.  First,  we 
must  understand  the  process  of  warning  analysis;  we  must  develop  an 
adequate  functional  model.  In  the  present  research  we  have  divided 
analysis  into  three  interrelated  stages  —  monitoring,  threat  recogni¬ 
tion,  and  projection  —  and  identified  a  number  of  procedures  and 
steps  associated  with  each  stage.  In  our  modeling  we  have  striven 
for  coherence.  Activities  in  the  monitoring  stage  are  intended  to 
reinforce  and  ramify  through  the  other  stages.  The  entire  process 
should  culminate  in  a  series  of  predictions  and  forecasts  abcut  the 
relative  likelihood  of  various  threats  and  situations. 
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From  our  perspective,  perhaps  the  major  pathology  in 
warning  analysis  occurs  when  the  human  analyst  fails  to  complete 
the  full  process  of  warning  analysis,  to  follow  through-  A  second 
pathology  would  be  exhibited  when  the  analyst  completes  the  steps 
but  fails  to  provide  appropriate  warning. 

These  two  major  pathologies  suggest  the  next  challenge  in 
pursuing  the  issue  of  effectiveness:  the  development  of  measures 
of  analytic  performance.  We  have  built  a  system  of  interconnected 
measures  of  analytic  effectiveness,  readiness,  and  maintenance, 
derived  by  monitoring  three  variables  associated  with  information 
and  analytic  dynamics:  backlog,  relat.ionality  (the  degree  to  which 
the  data  is  analyzed  in  relation  to  established  analysis  steps, 
procedures  and  methods);  and  signification  (the  assignment  of 
meaning  to  the  data).  We  cannot  summarize  the  measurement  system 
here,  but  do  note  that  we  have  distinguished  between  real  and 
estimated  effectiveness  in  warning.  One  can  see  the  relationship 
betwear.  the  problem  of  real  effectiveness  and  constraints  on  the 
human  mind  with  respect  to  post  mortems  and  hindsight  analysis. 

The  third  challenge  is  to  design  computer-aided  analysis 
methods  whose  enactment  creates  systematic,  thorough  analysis  as  we 
have  modeled  it  and  thereby  also  creates  the  measurement  data.  In 
very  simple  terms,  our  approach  is  this:  we  have  designed  the  struc¬ 
ture  of  a  working  context  for  warning  analysis  —  a  memory  of 
accumulated  analysis  —  largely  recorded  and  expressed  in  twelve 
forms  against  which  new  data  is  analyzed  for  meaning.  Highly 
changeable  through  a  machine-manipulated  configurational  language, 
these  forms  consist  of  classes  of  diagrammatic  structures  —  a 
language  of  modeling  —  well-suited  for  computer-driven  displays. 
There  is  great  flexibility  within  the  internal  structure  of  the  forms 
to  accommodate  the  idiosyncracies  of  individual  analytic  styles,  to 
allow  for  innovation,  and  also  to  permit  easy  registration  of  new 
meanings  and  rationale  as  changes  are  made  to  the  analytic  context. 
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Thus  within  the  language  of  the  forms,  the  individual  analyst  can 
perform,  record  and  display  his  particular  analysis.  The  twelve  forms, 
each  pertaining  to  a  different  point  in  the  warning  analysis  process, 
are  interrelated  and  progressive  —  a  logical  chain  of  outputs  of 
different  analytic  steps  and  procedures  —  and  thus  form  an  analytic 
language  with  a  syntax.  The  forms  are  used  through  an  analytic 
routine ,  an  itinerary  of  interrelated  analysis  steps  intended  to 
produce  rigor  whose  results  are  localized,  focused,  recorded  and 
displayed  in  the  context  of  the  forms.  The  routine,  which  embodies 
the  steps  and  procedures  in  the  model  of  analysis,  is  an  overall 
grammar  of  analysis.  The  performance  measures  are  implemented  simply 
by  using  the  computer-based  system.  When  forms  are  changed  in  certain 
ways  and  steps  in  the  routine  are  implemented,  the  associated  computer 
operations  and  other  activities  provide  the  needed  data  for  generating 
the  measures. 

6.  We  thus  arrive  at  the  following  perspective:  if  we 
concede  that  one  of  the  strongest  and  most  essential  impulses  in  man 
is  the  interpretive  impulse,  and  if  we  recognize  that  the  warning 
problem  is  among  the  most  serious  contemporary  problems  of  interpre¬ 
tation,  then  we  may  think  of  the  present  situation  as  one  in  which 
there  is  a  frustration,  a  blunting,  of  the  interpretive  impulse. 

We  can  appreciate  the  major  pathologies  of  analysis  —  the  case  in 
which  analysis  is  incomplete  and  the  case  in  which  complete  analysis 
is  inadequate.  We  can  think  especially  of  two  sources  of  constraints 
on  analysis.  One  consists  of  psychophysiological  obstacles  such  as 
those  discussed  above.  The  second  is  conceptual  in  nature.  This 
latter  obstacle  is  reflected  in  part  in  our  struggles  with  warning 
analysis  methodology,  particularly  in  the  projection  stage.  In  our 
view  the  second  obstacle  cannot  be  readily  overcome  until  tne  first 
obstacle  is  to  some  degree  mitigated:  until  the  cognitive  limita¬ 
tions  are  offset,  prescriptive  breakthroughs  will  be  that  much  more 


12 


IWAMS 


difficult.-  Overcoming  the  conceptual  limitations  will  be  accomplished 
largely  by  analysts  through  the  heuristic  process.  Computer-based 
support  appears  vital  to  that  process. 


Finally,  an  apology  is  due  the  reader  owing  to  the  consi¬ 
derable  expository  difficulty  of  attempting  to  convey  the  nature 
of  operations  on  IWAMS.  Given  our  goals,  it  is  incumbent  on  us  to 
convey  those  processes.  As  reported  below,  a  mock-up  version  of 
the  system  was  briefly  tested  informally  at  a  warning  center  late 
in  the  research  and  has  given  us  added  confidence  in  its  feasibility. 
Indeed,  as  we  discuss  below,  there  are  no  technical  obstacles  to  its 
rapid  development  and  implementation.  But  the  fact  remains  that  it 
is  difficult  to  convey  through  prose  and  static  diagrams  a  sense  of 
the  real  flow  of  analysis  by  an  experienced  analyst,  taking  cognizance 
of  the  analyst's  background  knowledge  and  the  IWAMS  high  context 
system,  and  assuming  interaction  with  computer-based  support  whose 
operation  the  analyst  has  mastered.  We  have  attempted  to  offset  this 
expository  difficulty  as  best  we  could  be  reminding  the  reader  when 
we  are  proceeding  painstakingly  and  slowly  through  processes  that  in 
actuality  would  be  far  different  in  speed  and  impact. 


We  may  now  turn  to  the  detailed  discussions  that  summarize 
the  research.  We  will  first  examine  the  model  of  warning  analysis. 
This  will  be  followed  by  a  detailed  discussion  of  the  system  of 
measures  of  warning  analysis  performance.  Following  that  is  a 
section  in  which  the  methods  of  warning  analysis  developed  in  the 
research  are  described  and  exemplified.  The  subsequent  section 
contains  a  description  of  the  software  design  for  the  IWAMS  system. 
This  is  followed  by  a  concluding  section  in  which  feasibility  and 
incentives  are  discussed. 
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The  first  task  is  to  develop  a  useful  model  of  warning 
analysis.  Given  the  encouraging  contemporary  progress  in  the  under¬ 
standing  and  modeling  of  human  cognition,  we  nevertheless  remain  far 
too  ignorant  to  make  any  pretense  of  modeling  all  activities  in  a 
mental  process  as  elusive  and  complex  as  warning  analysis.  We  must 
be  content  with  a  functional  model  which  identifies  major  stages, 
procedures  and  decision  points.  From  there  we  must  identify  crucial 
variables  in  the  cognitive  process  which  can  be  monitored,  change 
measurably,  and  which  are  diagnostic.  Such  a  model  is  essential  in 
developing  management  information  systems  yielding  analysis  perform¬ 
ance  measures . 

Better,  more  sophisticated  models  of  the  analysis  process 
constitute  a  goal,  a  hoped-for  benefit  from  gathering  empirical  data 
through  IWAMS.  The  present  model  is  a  start. 

Figure  2  shows  the  area  of  emphasis  within  the  overall 
IWAMS  concept  of  the  present  discussion. 

Elements  of  Warning  Analysis 

A  useful  functional  model  must  take  account  of  the  major 
elements  of  warning  analysis.  What  are  these  elements?  Thomas  Belden' 
view  of  warning  as  differing  from  other  intelligence  functions  because 
it  implies  decision  seems  a  fundamental  distinction  Beyond  that, 
several  definitions  of  warning  as  an  analytic  process,  a  category  of 
intelligence,,  and  a  type  of  intelligence  product  have  been  formulated. 
Various  organizations  and  individuals  have  made  definitions  (not 
always  in  agreement)  of  "tactical  warning",  "strategic  warning". 


Figure  2.  Modeling  Emphas 
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"indications  and  warning  intelligence",  "indicator",  "indication", 
"threat",  "indications  analysis",  etc. 

The  present  model  of  warning  analysis,  taking  into  account 
current  definitions,  is  intended  to  contribute  to  a  needed  compre¬ 
hensive  conceptual  framework.  At  minimum  the  authors  believe  that 
warning  analysis  shows  the  following  characteristics: 

•  It  involves  projections. 

•  It  uses  current,  estimative,  and  basic  intelligence. 

•  It  is  concerned  with  capabilities  and  intentions  of 
other  nations  and  entities. 

•  It  is  concerned  with  military,  political,  economic, 
and  cultural  dimensions. 

•  It  involves  a  time-critical,  watch-linked  current 
intelligence  orientation. 

•  It  also  involves  a  long-range,  nontime-critical, 
estimative  orientation. 

•  It  consists  of  a  set  of  discernible  stages,  one 
suitable  description  of  which  is:  monitoring,  threat 
recognition  and  projection. 

•  It  is  arguably  the  most  important  intelligence 
activity. 
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•  It  is  a  great  challenge,  there  being  enormous  epistem¬ 
ological,  cognitive,  institutional  and  other  constraints 
on  its  effectiveness. 

Captain  Frederick  Robitschek,  Jr.,  USAF,  has  developed  a 
useful  negative  definition  of  warning  analysis  which  complements  the 
above  list,  pointing  out  that  warning  is  not: 

•  Restricted  to  warning/forecasting  of  an  attack 

o  Restricted  to  evaluation  of  military  developments 

•  Restricted  to  events  with  a  direct  impact  on  the  U.S. 

and  its  allies 

•  Solely  the  purview  of  the  watch 

•  Limited  to  indications  analysis 

•  A  separate  and  arcane  branch  of  intelligence 

•  An  exercise  in  futility  -  impossible  to  produce. 

But  it  is  warning  analysis  as  a  cognitive  process  that  most 
concerns  the  authors.  The  model  of  this  process  described  below 
reflects  the  authors'  understanding  of  warning  analysis  as  observed 
during  protracted  visits  to  I&W  centers.  The  model  also  includes 
stages  and  procedures  not  necessarily  pursued  formally  now  in  the 
manner  described,,  but  which  analysts  and  manage!  s  have  concurred  with 
as  necessary  and  promising. 


ai  Sana. 
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Stages  of  Warning  Analysis 

Figure  3  shows  what  the  authors  believe  are  the  major 
progressive,  interrelated  functional  stages  in  warning  analysis: 
monitoring,  threat  recognition,  and  projection. 

The  monitoring  stage ,  outlined  in  Table  1,  may  be  viewed 
as  a  sequence  of  steps  and  procedures  beginning  with  the  review  of 
input  data  at  a  warning  center,  followed  by  the  sorting  of  the  data 
into  different  categories  of  interest,  a  comparison  of  the  data  with 
predetermined  indicators  whose  activity  is  judged  to  have  significance 
in  terms  of  emerging  threats,  an  identification  and  display  of  corre¬ 
lations,  a  preliminary  evaluation  and  specification  of  the  significance 
of  activity,  and  the  movement  of  the  results  to  the  next  stage  of 
analysis. 


Note  that  steps  8  and  9  comprise  a  safeguard  procedure 
against  analyst  tendencies  to  overlook  novel  situations.  The  analyst 
must  guard  against  cognitive  biases  and  problems  with  information 
which  may  impede  his  recognizing  the  onset  of  such  conditions.  As 
the  Soviets  are  inclined  to  point  out,  the  analyst  should  be  examining 
•'ambiguous”  and  "anomalous”  data  since  it  may  actually  signify  a 
threatening  situation  not  previously  well-imagined  and  modeled  by  an 
analytic  community.  The  problem  of  novel  threats  is,  of  course,  a 
fundamental  challenge.  Steps  8  and  9  begin  a  safeguard  routine 
within  the  analytic  procedures  which  develops  further  in  later  stages. 

The  input  data  to  an  I&W  center  will  include  several 
categories  of  message  traffic  from  various  collectors  and  a  variety 
of  bound  reports  which  originate  with  various  government  agencies  and 
departments.  The  message  traffic  can  contain  a  considerable  diversity 
of  military,  political,  economic  and  other  information.  The  addi¬ 
tional  input  may  be  in  the  form  of  daily  reports  on  various  subjects, 


Functional  Stages  in  Warning  Analysis 
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Table  1 .  Functional  Outline  of  Monitoring  Stage 


Step  1:  Review  input  data 
Step  2;  Sort  input  data 


Procedure 

1 

Procedure 

2 

Procedure 

3 

Procedure 

4 

Procedure 

5 

Procedure 

6 

Geographic 

Military 

Political 

Economic 

Cultural 

Other 


Step  3:  Match  sorted  data  against  predetermined 
indicators 


Step  4:  Identify  correlations 
Step  5:  Display  correlations 

Step  6:  Make  preliminary  evaluation  of  indicators 

Procedure  1;  Assess  relative  levels  of  activity 
Procedure  2:  Assign  significance  to  activities 


Step  7 :  Route  results  to  next  analysis  stage 
Step  8:  Review  residual  data 

Step  9;  Move  appropriate  residual  data  forward  for 
further  analytic  review 


d^aafaac 
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weekly  reports,  monthly  reports,  and  a  number  of  aperiodic  reports 
which  may  be  written  in  response  to  various  developing  situations. 

The  threat  recognition  stage  begins  where  the  monitoring 
stage  leaves  off.  The  steps  and  procedures  are  described  in  Table  2. 

The  analyst  now  progresses  from  matching  input  data  with 
individual  indicators  to  associating  active  indicators  with  threat 
situations  ("scenarios")  previously  modeled  in  some  chronological 
and  sequential  form.  This  is  a  higher  level  of  cognitive  challenge: 
some  indicators  —  at  early  enough  stages  perhaps  a  sizeable  portion 
of  them  —  may  seem  capable  of  being  associated  with  roughly  equal 
probabilities  to  more  than  one  possible  threat. 

The  future  cannot,  of  course,  be  precisely  modeled.  The 
fundamental  risk  that  novel  situations  will  occur  whose  shape  and 
dimensions  are  not  recognizably  analogous  to  models  currently  in  an 
analyst's  inventory  —  and,  indeed,  may  be  obscured  by  undue  focus 
on  existing  models  —  is  omnipresent.  Yet  models  abound.  They  have 
been  developed  in  many  forms  and  levels  of  detail  many  times  in  the 
history  of  intelligence  and  will  continue  to  appear.  Surely  all 
intelligence  organizations  of  any  magnitude  and  sophistication  have 
constructed  scenarios  depicting  how  adverse  actions  might  be  taken 
by  opponents.  Models  are  both  imperfect  and  a  valid  and  necessary 
response  to  the  challenges  facing  the  warning  analyst.  The  important 
ideas  are  that  models  be  viewed  as  approximations  and  treated  with 
appropriate  skepticism,  and  that  in  an  era  of  information  technology 
they  be  constructed  in  useful,  flexible  forms  (e.g.,  rapidly  display- 
able  and  changeable).  (In  subsequent  discussions,  considerable 
attention  will  be  focused  on  these  issues.) 
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Table  2 .  Functional  Outline  of  Threat,  Recognition  Stage 


Step  1:  Match  indicators  from  monitoring  stage  against 
predetermined  threat  models 

Procedure  1:  Call  up  preestablished  tnreat  models 

Procedure  2 :  Match  indicators  against  models 

a.  Call  out  critical  events/stages  (or  nodes)  of 
threat  models 

b.  Search  for  correlations  to  indicators 

c.  Identify  correlations 

d.  Review  correlations  against  other  critical 
events  whose  occurrence  has  been  previously 
detected  within  developing  situational  context 

e.  Assign  confidence  values  to  results  of  c  and  d 

f.  Identify  key  events  and  activities  whose 
occurrences  hav  i  not  been  detected 

g.  Identify  further  information  needs 

h.  Compare  all  preestablished  threat  models  to 
which  the  data  has  correlated 

(1)  Duplications 

(2)  Similarities 

(3)  Differences 

i.  Explore  for  possible  alternative  explanations/ 
hypotheses  for  data 

Step  2:  Move  pertinent  threat  models  to  projection  stages 
of  analysis 

Step  3 :  Conduct  novel  threat  analysis 

Procedure  1:  Formulate  new  threat  hypotheses 

Procedure  2 :  Display  threat  hypotheses 

Procedure  3:  Compare  residual  data  against  hypotheses 

Step  4:  Move  appropriate  novel  threat  models  to  projection 
phase  of  analysis 
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The  projection  stage  represents  the  final  and  by  far  the 
most  formidable  analytic  challenge.  The  analyst  now  surpasses  his  role 
as  historian  and  explores  the  future.  The  making  of  projections  is 
a  pervasive  and  highly  refractory  enterprise.  From  weather  forecasting 
through  human  resources  planning  to  economic  prognostication,  it  is 
rarely  done  to  the  satisfaction  of  the  forecaster  or  the  consumer  of 
such  forecasts.  Nonetheless,  we  confidently  advance  a  set  of  procedures 
for  threat  projection  that  seem  to  us  to  be  thorough,  easily  usable,  and 
useful  to  analysts.  This  set  of  procedures  is  described  in  detail  below 
in  Section  4. 

We  distinguish  four  types  of  projections  significant  in 
warning  analysis.  First,  projections  may  take  two  forms:  predictions 
and  forecasts ♦  By  our  definition,  prediction  involves  an  estimate  by 
analysts  of  conditions  necessary  for  given  situations.  However, 
predictions  are  not  necessarily  held  to  be  likely  (or  very  likely) 
by  analysts.  They  will,  however,  tend  to  address  situations  of  great 
interest  from  the  warning  viewpoint.  A  practical  view  of  predictions 
is  to  consider  them  a  vehicle  of  analytic  preparation.  In  the 
process  of  forming  and  refining  predictions,  the  analyst  thinks  about 
(indeed,  analyzes )  various  existing,  imminent  and/or  remote  condi¬ 
tions  whose  occurrence  could  create  threats. 

The  term,  forecast,  is  used  to  refer  to  a  projection  in 
which  an  analyst  estimates  that  the  event  or  situation  will  occur. 

Hence  a  forecast  may  be  thought  of  as  a  prediction  which  the  analyst 
now  believes  will  in  fact  take  place  with  an  implicit  or  explicit 
probability  attached. 

Further,  boch  predictions  and  forecasts  mr.y  be  of  two 
types:  unimpeded  and  influenced.  An  unimpeded  projection,  whether 
a  prediction  or  a  forecast,  is  one  in  which  the  analyst  judges  that 
under  the  current  friendly-side  posture,  the  projected  situation  is 
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likely  to  occur  in  a  given  form.  In  other  words,  the  analyst  judges 
that  unless  U. S ./friendly  decisionmakers  intervene  and  change  condi¬ 
tions  over  which  they  are  presumed  to  have  control,  the  projected 
outcome  is  likely  to  take  a  certain  form,  designated  the  unimpeded 
projection.  The  influenced  projection  is  one  in  which  the  analyst 
considers  the  impact  of  -srtain  activities  which  his  decisionmaker 
may  introduce  and  their  impact  on  the  outcome.  This  becomes  the 
influenced  projection. 

Table  3  (see  page  26)  is  an  outline  of  functional  procedures 
and  steps  in  the  projection  stage.  Most  of  the  procedures  are  self- 
explanatory  (and  will  be  elaborated  on  in  later  discussions  of 
methodology),  but  comments  are  appropriate  for  procedures  1-3  under 
step  1.  For  each  of  the  four  types  of  projection,  the  first  proce¬ 
dure  entails  an  important  transformation:  the  analyst  translates  the 
individual  threat  models  from  the  preceding  stage  into  a  predictive 
format  organized  according  to  the  basic  information  categories  — 
who,  what,  where,  how,  when,  and  why.  In  our  judgment,  projections 
should  ultimately  be  expressed  in  a  very  fundamental  grammar,  namely, 
the  information  categories  linked  together  in  English  sentence  struc¬ 
ture.  This  grammar  of  projections  can  accommodate  simple  or  complex 
projections.-  (Examples  are  given  below  of  the  transformation  of 
threat  models  into  this  format. ) 

The  second  step  entails  comparing  the  specificities  of 
individual  threat  projections.  A  Specificity  Rating  Scale  by  which 
analysts  may  estimate  the  degree  of  specificity  for  each  of  the 
information  categories  in  any  given  threat  projection  has  been 
developed  and  is  described  below.  Predictions  and  forecasts 
expressed  formally  in  terms  of  who,  what,  when,  where,  how,  and  why 
may  be  compared  quantitatively  in  each  category.  For  example,  the 
category,  when,  may  be  highly  specific  (described,  say,  to  the  hour) 
or  extremely  general. 
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The  third  procedure  for  each  type  of  projection  is  to 
develop  and  elucidate  the  rationale.  This,  of  course,  is  the  greatest 
challenge  to  the  analyst,  the  most  difficult  single  analytic  challenge 
in  the  entire  process  of  performing  warning  analysis.  The  analyst 
must  analyze  the  conditions  that  might  lead  to  various  threat  enact¬ 
ments  and  assess  the  relative  importance  and  interdependence  of  the 
conditions.  The  conditions  could  include  military  capabilities, 
political  aspects,  economic  factors,  social  and  cultural  conditions 
and  forces,  decisionmaker/leadership  perceptions  and  predispositions, 
and  other  key  variables.  This  third  procedure  must  receive  the 
greatest  emphasis  in  methodology. 


The  Model  and  Current  Operations 

It  is  not  within  the  present  scope  to  treat  this  topic 
comprehensively,  but  some  comments  are  useful. 

Formal  procedures  and  supporting  technology  for  the  moni¬ 
toring  stage  are  evidenced  in  current  operations.  Procedures  for 
systematically  reviewing  input  data  against  preestablished  indicators, 
and  reporting  results,  are  well  established.  Various  computer-based 
systems  for  quantifying  aspects  of  indicator  activity  are  being 
explored  and  tested  operationally.  Some  of  these  are  described  below. 

In  terms  of  threat  recognition  and  projection,  however,  less 
formal  structure  has  been  created  to  date.  We  are  just  beginning  to 
see  greater  focus  being  placed  on  the  issue  of  analytic  rigor.  Plans 
exist  now  for  further  research  in  threat  modeling  (e.g.,  modeling  past 
crises  as  aids  to  warning  analysts)  and  in  projection  methodologies. 
Tims  some  momentum  is  being  built  up. 

Importantly,  emphasis  on  automatic-data-processing  support  to 
analysts  is  also  generally  increasing.  The  ability  to  perform  the  kind 
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of  thorough  and  comprehensive  analysis  implicit  in  the  model 
described  above  —  and  the  prospects  for  rapidly  evolving  refinements 
in  analytic  procedure  —  are  crucially  related  to  design  and  develop¬ 
ment  of  useful  computer-based  analytic  support  technology. 

Themes  Sounded  in  the  Discussion  of  the  Model 


Several  very  basic  themes  in  the  warning  problem  have  been 
sounded  which  will  be  discussed  further  in  subsequent  sections.  These 
include:  limitations  in  understanding  and  modeling  human  cognition; 
cognitive  and  other  obstacles  to  warning  analytic  effectiveness;  the 
relationship  of  information  technology  to  advanced  warning  analysis; 
areas  of  conceptual  confusion;  the  methodological  crisis;  and  the 
problems  of  monitoring  and  managing  analysis.  A  separate  theme  is 
the  expository  difficulty  in  conveying  the  actual  dynamics,  excite¬ 
ment  and  incentive  in  performing  complete  warning  analysis,  as  modeled 
above.  But  the  major  theme  is  this:  the  primary  pathology  of  warning 
analysis  is  the  failure  to  complete  the  full  process  of  analysis. 

All  these  issues  —  and  others  as  well  —  are  at  the  heart 
of  the  next  problem  we  must  confront:  the  development  of  analysis 
performance  measures. 
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Table  3.  Functional  Outline  of  Projection  Stage 


Step  1:  Compare  candidate  threats  in  unimpeded,  predictive 
domain 

Procedure  1:  Translate  individual  threats  into 

predictive  format  based  on  information 
categories 

a.  Who  (or  what) 

b.  (Could  do)  what 

c.  (to)  whom  (or  what) 

d.  Where 

e.  How 

f .  When 

g.  Why  (or  because  X  conditions  apply) 

Procedure  2 :  Compare  specificities  of  individual  threat 
predictions 

Procedure  3:  Elucidate  rationale  behind  specificities 

a.  Estimate  conditions  that  would  support  various 
threat  enactments,  and  their  relative 
importance  and  interdependence 

(1)  Military  capabilities 

(2)  Political  situation 

(3)  Economic  conditions 

(4)  Social/cultural  conditions  and  forces 

(5)  Decisionraaker/leadership  perceptions  and 
predispositions 

(6)  Other 

b.  If  conditions  now  hold,  estimate  earliest 
possible  occurrence  in  future  of  threat 
enactment 

c.  Identify  any  missing  conditions 

d.  For  missing  conditions,  estimate  earliest 
occurrence  in  future 
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Table  3.  —  Continued. 


e.  Estimate  earliest  point  in  future  pertinent 
threats  could  occur  after  missing  conditions 
occur. 

Procedure  4:  Indicate  sources  (e.g.,  outside  expertise) 

Procedure  5:  Indicate/summarize  assumptions 

Procedure  6:  Indicate/summarize  uncertainties  and 
data  requirements 

Procedure  7:  Across  predictions,  and  added  to  previous 
comparisons,  isolate  and  identify  by 
information  category  the  following 

a.  Duplications 

b.  Similarities 

c.  Differences 

Procedure  8:  Compare  predictions  in  terms  of  proba¬ 
bility  against  selected  uimelines 

2:  Compare  candidate  threats  in  influenced  predictive 
domain 

Procedure  1:  Consider  potential  U.S.  influences  on 
development  of  situations 

a.  Identify  U.S.  influence  options 

Procedure  2:  Translate  influenced  threat  models  into 
predictive  format  (as  in  step  1  above) 

Procedure  3 :  Repeat  rest  of  procedures  in  step  1 

Procedure  4:  Make  grand  comparison  of  unimpeded  and 
influenced  predictions. 


_3:  Perform  similar  procedures  for  unimpeded  and 
influenced  forecasts. 
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Two  points  should  made  at  the  outset.  First,  we  will 
explore  and  develop  measures  of  warning  analysis  without  direct 
concern  about  how  they  would  be  implemented.  The  primary  purpose 
at  this  point  is  a  conceptual  analysis  of  the  measurement  problem. 
Implementation  is  described  below  in  detail  in  Sections  4  and  5 . 

Second,  the  overall  system  of  measures  should  be  previewed. 

It  is  a  comparatively  simple  system.  There  are  three  major  (and 
classic)  measures:  analytic  effectiveness ,  readiness ,  and  maintenance . 
These  measures  are  obtained  essentially  by  monitoring  three  crucial, 
interrelated  variables  in  the  flow  of  information  through  the  stages 
and  procedures  of  warning  analysis:  data  backlog,  data  relationality, 
and  data  signification.  This  in  turn  creates  the  opportunity  to 
monitor  a  number  of  other  variables. 

The  matrix  below  shows  the  essential  relationship  of  the 
measures  to  the  model  of  warning  analysis . 


Model 

Measures 

Effectiveness 

Readiness 

Maintenance 

Monitoring 

Threat  Recognition 

Projection 

The  remainder  of  the  present  section  will  discuss  in  detail  the 
definition  of  the  measures  and  the  development  of  a  measurement 
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system.  Figure  4  shows  the  area  of  emphasis  in  the  present  discus¬ 
sion  with  respect  to  the  overall  IWAMS  concept. 

The  Conceptual  Framework 

Development  of  the  measures  proved  one  of  the  most  difficult 
challenges  in  the  research.  Little  previous  research  seemed  directly 
applicable.  Yet  measuring  an  elusive  cognitive  process  of  rationality 
and  intuition  requires  building  a  conceptual  framework.  We  shall 
sketch  in  this  framework  as  background  to  discussion  of  the  measures. 

•  The  Essential  Dynamic  of  Analysis.  Analysis,  a 
cognitive  process,  is  described  in  terms  such  as  "association", 
"interpretation",  "meaning,"  "imagination,"  "intuition,"  "rationale," 
"induction",  "deduction",  etc.  In  analysis,  essentially  we  associate 
different  data  and  assign  meaning.  Because  the  analyst  makes  projec¬ 
tions,  warning  analysis  heavily  involves  the  imagination. 

A  major  point  is  the  importance  of  the  information 
throughput.  Given  that  input  data  is  needed  for  the  analytic  dynamic 
to  occur,  and  that  warning  analysis  takes  place  when  the  analyst 
reviews  the  data  and  assigns  meaning,  this  process  occurs  throughout 
the  analytic  stages  as  the  data  is  throughput .  The  basic  dynamic 
can  be  depicted  as  follows: 


MEASURES  DYNAMIC 


MONITORING 

THREAT 

RECOGNITION 

PROJECTION 

INFORMATION  THROUGHPUT 


\ 
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The  essential  measurement  strategy  is  to  identify  key  variables 
within  the  analytic  functions  in  each  of  the  three  stages  and  then 
design  a  means  of  monitoring  the  variables. 

•  Perspectives  From  Living  Systems  Theory.  Portions  of 
James  Grier  Miller's  work  on  the  theory  of  living  systems  have  been 
useful  in  developing  a  conceptual  framework  for  measures  development. 
Miller  views  living  systems,  ranging  from  cells  to  supranational 
organizations,  as  open  systems  composed  of  subsystems  which  process 
inputs,  throughputs,  and  outputs  of  various  forms  of  matter,  energy 
and  information.  He  identifies  six  information  subsystems  as  common 
across  the  range  of  living  systems:  the  input  transducer,  the 
decoder,  associative  functions,  memory,  the  decider,  and  the  encoder. 
The  internal  transducer  is  the  sensory  subsystem  which  receives 
markers  from  subsystems  or  components  within  the  system  bearing 
information  ab-_ut  significant  alterations  in  those  subsystems  or 
components .  Association  and  memory  are  linked  in  representing  the 
two  basic  elements  of  the  learning  process  within  a  system.  The 
decider  makes  judgments  about  the  significance  of  information.  The 
decoder  takes  input  information  and  translates  it  into  an  internal 
or  private  code  useful  within  the  system.  The  encoder  translates 
the  final  output  into  what  might  be  designated  a  public  code  for 
dissemination. 

Vie  .-implicitly  have  been  viewing  warning  analysis  as  cogni¬ 
tion  within  a  general  system  through  which  data  progresses  and  whereby 
perspective  is  creased.  Figure  5  associates  our  warning  model  with 
some  of  the  structure  of  living  systems  theory.  Such  theory  can  aid 
us  in  deriving  useful  categories  and  structure  for  exploring  the 
problem  of  analytic  measures.  In  sum,  warning  analysis,  a  process 
involving  people,  information,  software,  hardware,  and  other  elements, 
may  be  thought  of  as  a  system  with  various  measurable  states  of 
activity,  some  appropriate  and  some  to  be  avoided. 


Figure  5.  Warning  Analysis  and  Elements  of  Living  Systems  Structure 
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Figure  5  lists  some  key  variables  in  information  systems, 
some  classic  pathologies  in  such  systems,  and  typical  adjustments  to 
the  pathologies.  In  subsequent  discussions  of  specific  warning 
measures,  portions  of  Miller's  living  systems  structure,  particu¬ 
larly  the  variables,  will  be  explored  further. 

•  Perspectives  From  Information  Theory.  From  infor¬ 
mation  theory  we  must  consider  the  problem  of  meaning,  for  meaning 
is  central  to  the  measurement  of  warning  analysis.  We.  have  developed 
the  ability  to  measure  rates  of  information  as  it  throughputs  systems. 
It  is  the  measurement  of  meaning  that  has  eluded  us,  for  meaning 
cannot  be  objectively  quantified  since  it  involves  a  human  observer, 
a  human  participant,  and  hence  subjectivity.  Meaning  may  be  defined 
as  follows: 

Meaning  represents  the  significance  of  information  to  the 
system  which  processes  it.  Meaning  constitutes  a  change  in 
that  system's  processes  elicited  by  the  information  input, 
often  resulting  from  associations  made  to  it  on  previous 
experience  with  it.  Furthermore,  this  change  can  occur 
immediately  or  later. 

Another  crucial  concept  is  entropy  in  its  relationship  to 
information.  Information,  taken  as  a  retained  model  of  reality, 
will  suffer  entropy  over  time.  In  warning  analysis,  scenarios  of 
possible  threats  (e.g.,  from  North  Korea,  U.S.S.R.,  Libya,  etc.) 
obviously  must  be  updated  periodically.  Models  are  static  approxi¬ 
mations  of  a  reality  that  is  always  changing  in  manifold  ways.  The 
key  concept  for  the  measurement  problem  is  that  new  information  can 
counteract  entropy. 


•  Perspectives  From  Learning  Theory.  A  fundamental 
issue  is  the  relationship  among  input  data,  association,  memory, 
and  imagination:  the  learning  process  of  a  warning  analyst  as  it 
entails  mental  operations  on  data.  The  discussion  of  analytic 
technique  and  methodology  (Section  4)  in  part  involves  the  learning 
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process,  especially  as  supported  by  advanced  technology  —  computer- 
based  extrasomatic  memories  which  represent  collectively  created, 
coherent  records  of  analysis  immune  to  obscurring  processes  of  the 
human  memory. 

•  Perspectives  from  Cybernetics.  Cybernetics  obviously 
is  at  the  core  of  the  present  research.  Since  we  are  designing  a 
management  system,  we  have  emphasized  the  basic  concept  of  feedback 
control,  and  this  is  implicit  at  all  points  in  the  discussion. 


Basic  Measures 


Three  basic  measures  of  warning  analysis  are  effectiveness, 
readiness,  and  maintenance .  Analytic  effectiveness  is  an  outcome,  a 
result.  Analytic  readiness  is  a  state,  a  condition.  Analytic 
maintenance  is  an  activity.  The  diagram  below  is  meant  to  reinforce 
the  interrelated,  interdependent  nature  of  the  measures. 
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Figure  6  shows  a  simplified  concept  of  a  hierarchy  of 
the  measures  and  supporting  variables.  Effectiveness,  readiness  and 
maintenance  (E,  R,  M)  are  derived  by  monitoring  certain  information- 
related  variables  which  can  be  thought  of  in  hierarchical  terms.  For 
example,  data  backlog  at  various  stages  in  the  analysis  will  become 
an  important  variable,  and  in  turn  will  be  associated  with  other 
information-related  "subvariables"  such  as  percentages  of  informa¬ 
tion  by  type  (for  example,  military,  economic,  political,  COMINT, 
ELINT,  etc.).  Many  combinations  of  variables  might  be  monitored 
and  analyzed.  An  IWAMS  design  goal  is  to  insure  that  enough  data 
of  interest  is  collected  to  allow  multiple  correlations  among 
different  variables  in  the  future,  including  correlations  whose 
interest  was  not  anticipated  earlier.  As  the  remarks  on  living 
systems  theory  suggested,  certain  key  variables  are  associated  with 
information  processing  systems  at  all  levels  and  should  be  accounted 
for  in  the  design  structure. 

Figure  6  also  identifies  important  variables  which  are 
not  exclusively  information-related,  such  as  the  dimensionality  of 
the  threat  models  and  the  use  of  anti -bias  analysis  procedures. 

Finally,  the  diagram  below  conveys  the  IWAMS  design 
principle  that  all  three  measures  should  be  monitored  at  each  major 
stage  of  analysis.  It  is  implicit  that  the  measures  must  ultimately 


M.R,E  Art  7  DERIVED  FOR  EACH  -TAGE/SUBST AGE  OF  ANALYSIS.  CUMULATIVE  M,P„E  ALSO  DERIVED 
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reach  the  steps  and  procedures  within  each  stage.  Further,  cumula¬ 
tive  values  must  be  obtained  by  integrating  measurement  data  across 
and  among  the  various  stages  to  foster  diagnosis. 

These  points  have  established  a  context  for  the  detailed 
discussion  of  the  development  of  measures  to  which  we  now  turn. 


Effectiveness 


By  far  the  most  complex  and  difficult  measure  is  effective¬ 
ness.  Effectiveness  is  a  measure  of  the  ability  to  provide  accurate, 
specified  warning  in  a  timely  manner.  It  differs  from  readiness  and 
maintenance  as  follows:  Readiness  is  a  measure  of  the  ability  to 
carry  out  existing  analytic  procedures  and  methodology.  Maintenance 
concerns  the  activity  by  which  analytic  procedures,  models,  and 
memories  are  sustained  against  entropy,  kept  current,  and  rendered 
operational . 

The  important  point  about  the  effectiveness  measure  is 
that  there  are  really  two  kinds  of  warning  analysis  effectiveness: 
estimated  effectiveness  and  real  effectiveness .  The  warning  analyst 
confronts  the  future.  Real  effectiveness  must  be  verified  after  the 
fact;  validation  is  delayed,  occurring  through  the  post  mortem.  Yet 
there  is  a  trap:  hindsight  analysis  is  perilous.  There  are  sweeping 
arguments  against  the  validity  of  historical  reconstruction  itself: 
under  what  conditions  can  warning  be  reliably  connected  causually 
to  an  outcome?  Skeptics  of  post  mortems  on  intelligence  failures 
cite  problems  in  validly  reconstructing  past  meanings  of  information 
to  analysts  and  analytic  communities,  which  is  indeed  a  very  critical 
problem  we  must  deal  with  subsequently. 
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The  diagram  now  shows  the  analysis  measures  expanded  to 
include  real  effectiveness. 


Estimated  effectiveness,  an  a  priori  measure,  is  based  on 
assumptions.  These  include  the  assumption  that  usually  multidimen¬ 
sionality  is  good  in  warning  analysis:  that  it  is  better  to  take 
into  account  several  frames  of  reference  (political  and  economic  and 
military,  etc.)  m  interpreting  a  situation  than  only  one.  Another 
major  assumption  is  that  warning  analysis  ought  to  incorporate  safe¬ 
guard  procedures  designed  to  offset  the  inevitable  epistemological 
constraints  and  human  tendencies  toward  bias  which  lead  to  suscep¬ 
tibility  to  deception,  both  natural  and  instigated  by  adversaries. 
Estimated  effectiveness  therefore  becomes  a  wager  that  thoroughness, 
rigor,  and  systematic  approach  in  the  methods  of  performing  analysis 
will  lead  to  greater  real  effectiveness. 
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Basic  Obstacles  to  Effectiveness.  The  antithesis  of  warning 
is  surprise.  Effectiveness  in  warning  analysis  must  ultimately  be 
measured  in  the  context  of  avoiding  surprise.  The  authors  believe 
there  is  a  shortfall  in  prescriptive  research  on  mitigating  surprise, 
although  fascinating,  useful  discussions  of  problems  in  warning 
analysis  have  been  made  in  Wohlstetter ! s  study  of  Pearl  Harbor, 
Schlaim's  study  of  warning  failures  in  the  Middle  East,  Whaley’s 
studies  of  deception,  and  Chan’s  work  on  warning  failures.  Outside 
the  literature  of  warning  there  are,  of  course,  many  discussions 
of  basic  epistemological  and  cognitive  constraints  on  man's  ability 
to  analyze  the  past,  present,  and  future. 

Certainly  warning  analysis  pursued  energetically  is  a  hard 
business,  a  struggle.  The  analyst  might  liken  himself  to  Sisyphus, 
condemned  forever  to  attempt  to  push  a  heavy  rock  up  and  over  a  hill, 
forever  to  be  defeated  near  the  top  by  insurmountr tie  inertia.  Most 
basically,  the  analyst  is  implicitly  striving  for  literal  realism  in 
a  bewildering  reality  which  does  not  necessarily  arrange  itself 
according  co  his  mental  constructs  and  images.  Not  that  he  has  a 
choice  in  any  final  sense.  He  will  do  what,  in  Joan  Didion's  words, 
all  humans  do: 

We  tell  ourselves  stories  in  order  to  live  we  interpret 
what  we  see,  select  the  most  workable  of  the  multiple  choices.; 

We  live  entirely  , ..«  by  the  imposition  of  a  narrative  line 
upon  disparate  images,  by  the  'ideas'  with  which  we  have 
learned  to  freeze  the  shifting  phantasmagoria  which  is  our 
actual  experience. 

As  we  shall  discuss  later,  there  is  some  degree  of  kinship 
between  the  analyst  and  journalists  and  writers;  between  epistemo¬ 
logical  crij.es  in  the  arts  and  fundamental  problems  in  warning 
analysis..  There  is  less  distance  between  concepts  such  as  narra~ 
tology  and  the  problem  of  warning  analysis  than  might  at  first  seem 
the  case.  The  analyst,  after  all,  seeks  to  interpret  reality,  to 
impose  order  on  the  flux  of  events.  For  example,  the  impulse  to 
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model  possible  threatening  situations  comes  from  the  analyst's  need 
to  reduce  a  bewildering  flux  of  events  to  some  orderly  system.  Of 
late  there  is  an  announced  crisis  of  confidence  in  epistemological 
terus  on  the  part  of  some  journalists,  novelists  and  other  artists, 
centered  around  issues  of  nonfiction  versus  fiction  novels;  the 
problem  of  interpretation  in  journalism;  and  other  aspects  in  turn 
related  to  the  perversity  of  reality  in  relation  to  our  interpretive 
models  and  constructs .  At  the  heart  of  the  problem  of  analytic 
effectiveness  is  the  problem  of  modeling  a  complex,  multifaceted 
reality  greater  than  our  ability  to  perceive  it.  Our  organizing 
models  of  reality  can  only  be  approximations. 

Besides  making  sense  of  current  situations  (and  usually 
remote  ones),  the  analyst  confronts  and  must  explore  the  future.  Of 
course  very  few  events  now  are  actually  projectable  by  man  with  great 
specificity.  Using  the  example  of  attempting  to  project  the  activity 
of  a  human  being,  Edward  0.  Wilson  writes: 


Fven  though  human  behavior  is  enormously  complicated  and 
variable  ....  theoretically  it  can  be  specified.  Genetic  con¬ 
straints  and  the  restricted  number  of  environments  in  which 
human  beings  can  live  lm.\t  the  array  of  possible  outcomes 
substantially.  But  only  techniques  beyond  our  present 
imagining  could  hope  to  achieve  even  the  short-term  predic¬ 
tion  of  the  detailed  behavior  of  an  individual  human  being, 
and  such  an  accomplishment  might  be  beyond  the  capacity  of 
any  conceivable  intelligence.  There  are  hundreds  or  thou¬ 
sands  of  variables  to  consider,  and  minute  degrees  of 
imprecision  in  any  one  of  them  might  easily  be  magnified  to 
alter  the  action  of  part  or  all  of  the  mind.  it  may  be 

a  law  of  nature  that  no  nervous  system  is  capable  of  acquiring 
enough  knowledge  to  significantly  predict  the  future  of  any 
other  intelligent  system  in  detail. 

Problems  in  projection  are  compounded  when  one  considers  the 

challenge  of  projecting  changes  in  large-scale  systems. 
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Given  that  warning  projections  must  be  imperfect  in  the 
above  sense,  a  major  strategic  aspect  of  warning  analysis  becomes 
more  apparent.  The  warning  analyst  will  serve  a  valuable  role  by 
alerting  the  command  sufficiently  that  it  may  in  turn  take  action 
designed  to  preclude  the  possibility  of  an  adverse  situation  and 
outcome.  The  distinction  made  above  between  unimpeded  and  influ¬ 
enced  projections  implies  this  strategy. 

Given  the  severe  constraints  on  warning  analysis,  a  natural 
approach  to  developing  effectiveness  measures  is  to  begin  with  classi¬ 
fication  of  the  specific  obstacles  to  analytic  effectiveness,  and  then 
to  conceive  of  measures  which  bear  on  capabilities  and  strategies  for 
overcoming  the  obstacles. 

Epistemological  Obstacles .  Several  generic  difficulties 
can  be  identified: 

«  Limited  Availability  of  Key  Data.  Information  is  a 
model  of  the  real  world.  By  definition,  information 
will  be  limited  with  respect  to  total  reality.  Key 
data  may  not  be  available  at  a  given  time  to  the 
observer. 

•  Analyst  Limitations  in  Experience  and  Knowledge .  A 
given  analyst's  own  knowledge  and  experience  may  not 
connect  with  given  situations.  In  Soviet  literature, 
the  example  is  given  of  a  Japanese  commander  fighting 
on  a  remote  Pacific  island  in  World  War  II  who 
refuses  to  believe  reports  of  the  atomic  bomb?ng  of 
Hiroshima  and  Nagasaki.  The  Commander  has  no  real 
concept  of  the  development  and  potential  destructive 
applications  of  atomic  power  in  that  period  and  is 
caught  up  on  the  local  combat.  He  has  no  appropriate 
context  or  frame  of  reference. 
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•  Novel  Threat  Situations.  A  serious  problem  in  warning 
analysis,  the  novel  threat  situation  seems  novel 
because  its  characteristics  have  (or  appear  to  have) 
little  or  no  historical  antecedents.  Its  features 
say  appear  innocent  or  go  unnoticed  for  periods. 

#  Similarity  in  Situations.  Some  situations  may  be 
difficult  to  recognize  because  of  their  similarity  in 
early  stages  to  other  possible  situations.  For  example, 
when  analysts  develop  a  set  of  threat  models,  it  i3  not 
unlikely  that  the  early  events  postulated  in  various 
■odels  will  be  similar. 

e  Numerous  Possible  Situations.  Since  there  are  so  many 
variations  of  possible  situations,  it  is  always  some¬ 
what  futile  (if  necessary)  to  attack  the  recognition 
problem  by  assembling  a  priori  catalogs  of  events 
presumed  typical  of  threat  conditions.  The  classic 
pattern  recognition  problem  is  a  good  deal  simpler: 
a  closed  alphabet  of  patterns  is  assembled  and  the 
viewer  must  essentially  sort  out  the  patterns  which 
are  throughput.  There  are,  however,  effectively  an 
infinite  number  of  variations  in  real  situations,  so 
that  an  analyst  must  view  preestablished  threat  models 
with  appropriate  skepticism. 

Without  pushing  too  far  the  distinction  between  "cognitive1' 
and  “epistemological , ■  it  is  useful  to  discuss  certain  cognitive 
obstacles: 


•  Restricted  Perspective.  As  Schlaim  in  particular 
discusses,  analysts  sometimes  tend  to  work  within 
narrow  hypotheses.  This  is  certainly  understandable, 
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for  one  can  provide  only  so  many  models  of  the  future. 
Constructing  such  models  (or  scenarios)  of  real  world 
situations,  in  both  the  arts  and  sciences,  is  one  of 
the  most  challenging  and  exhausting  endeavors  we  can 
undertake.  Further,  once  they  are  constructed  there 
is  a  psychology  of  preservation  powerfully  at  work. 

One  must  also  believe  that  the  sense  of  imperfection 
about  the  models  can  at  times  diminish  the  will  to 
struggle  with  them.  In  short,  hypotheses  may  be  hard 
won  and  defended  stubbornly. 

•  Bias .  A  related  problem  is  bias,  which  is  seen  in 
tendencies  to  establish  low  thresholds  for  input  data 
that  confirm  prior  assumptions  and  high  thresholds 
for  input  data  that  may  suggest  other  explanations. 

•  Methodological  Inflexibility.  Analysts  may  tend  to 
work  with  only  one  or  two  "proven"  techniques  of 
analysis,  rather  than  selecting  from  a  variety  of 
methodological  alternatives  in  an  effort  to  fit 
technique  to  problems.  One  basic  assumption  about 
effective  analysis  is  that  it  is  better  to  analyze 
information  from  several  viewpoints,  using  a  variety 
of  techniques.  The  premise  is  that  one  of  the 
analyst's  defenses  against  the  complexity  of  reality 
is  to  search  for  meaning  by  taking  several  cognitive 
paths . 

•  Susceptibility  to  Deception.  The  problems  of  warning 
analysis  can  be  aggravated  by  the  adversary  as  deceiver. 
Clever  deception  will  try  to  exploit  epistemological 
and  cognitive  weaknesses.  The  world  itself  deceives 
the  human  observer  easily  enough;  clever  deception 
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operations  can  raise  further  the  challenge  to  the 
warning  anal  Tst. 

Experimentation  has,  of  course,  demonstrated  much  about 
cognitive  biases.  Psychologists  have  charted  limitations  in  man's 
memory,  attention  span,  reasoning  capability  and  other  cognitive 
functions.  These  limitations  considerably  affect  man's  ability  to 
process  data  to  arrive  at  judgmental  decisions,  estimates,  etc.  As 
we  have  said,  the  limitations  result  in  part  from  the  complexity  of 
our  environment:  it  forces  us  to  adopt  simplifying  strategies  of 
perception,  comprehension,  inference,  and  decision.  These  remarkably 
adaptive  strategies  allow  us  to  deal  with  the  multiplicity  of  infor¬ 
mation  we  take  in  and  must  process;  yet  ironically  they  result  often 
in  erroneous  judgment.  Thomas  Beldon  has  denoted  one  consequence  of 
cognitive  biases  in  intelligence  analysis  as  the  condition  of 
"hardening  of  the  categories". 

Preliminary  Concepts  of  Effectiveness  Measures.  We  can 
now  perceive  concepts  of  possiole  measures  of  effectiveness.  A  few 
examples  are: 

•  Ability  to  monitor  on  the  basis  of  integrated  indica¬ 
tors  (military  plus  political  plus  economic). 

•  Existence  and  use  of  analytic  safeguards  to  offset 
epistemological  and  cognitive  problems  m  warning, 
such  as : 

Threat-search  procedures  for  avoiding  the  trap  of 
overlooking  threats  not  previously  imagined  and 
modeled,  e.g.,  techniques  for  learning  from 
"ambiguous'1  and  "incomplete"  information. 
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Procedures/techniques  for  weighing  opposing 
hypotheses . 

Procedures  that  foster  systematic  use  of  a  mixture 
of  analytic  approaches  (e.g.,  various  forecasting 
and  prediction  techniques)  to  increase  objectivity 
and  offset  the  inevitable  bias  resulting  from 
overemphasis  on  any  one  approach  or  type  of  data. 

Ability  to  perform  easily  the  functions  keyed  to 
estimation  and  dissemination,  such  as:  marshall 
evidence;  develop  statistical  presentations; 
argue  by  precedent,  individual  indicators,  and/or 
patterns  of  indicators;  and  use  of  modeling 
techniques . 

Especially  when  we  discuss  information-related  variables, 
we  shall  resume  the  exploration  and  development  of  concepts  of 
measures  of  analytic  effectiveness.  It  is  useful  now,  however,  to 
proceed  to  a  discussion  of  the  readiness  measure. 

Readiness 

Readiness  has  been  characterized  as  a  state,  a  condition, 
a  measure  of  the  ability  to  undertake  the  procedural  and  mevhodo- 
logical  activity  within  warning  analysis.  Readiness  applies  to  all 
the  stages,  steps  and  procedures  within  the  model  of  analysis;  it 
must  be  measurable  for  any  and  all  of  them. 

At  the  beginning  we  need  to  examine  an  important  aspect  of 
the  relationship  of  readiness  to  effectiveness.  We  have  discussed 
the  problems  of  imperfect  effectiveness  in  warning  analysis;  that 
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the  sources  of  imperfection  are  epistemological  and  human  cognitive 
problems.  In  an  important  sense,  readiness  is  subordinate  to,  depen¬ 
dent  upon,  effectiveness.  One  can  develop  a  system  of  analytic 
methods  that  entails  certain  procedures,  techniques  and  methodology 
designed  to  foster  analytic  effectiveness,  and  upon  which  readiness 
can  be  based.  The  whole  analysis  system,  however,  must  be  imperfect 
in  the  face  of  reality.  It  cannot  guarantee  success. 

lhe  significant  point  is  this:  analysts  in  analytic  groups 
must  therefore  be  said  to  be  in  high  (or  low)  states  of  readiness  to 
follow  an  imperfect  system  of  analysis.  The  effectiveness-readiness 
relationship  bears  on  a  number  of  aspects  of  the  present  research. 

For  example,  consider  the  use  to  whicn  measurement  data  might  be  put 
in  a  post  mortem  with  respect  to  the  issue  of  an  "intelligence 
failure."  Given  that  analysts  are  working  with  imperfect  techniques, 
the  obligation  to  be  effective  must  be  defined  within  conditional 
terms;  the  obligation  for  readiness,  however,  may  be  viewed  indepen¬ 
dently  of  the  question  of  real  effectiveness. 

There  is  also  an  important  relationship  between  readiness 
and  maintencince .  Readiness  is  a  state  which  derives  from  maintenance 
activity,  such  as  the  activity  of  keeping  the  information  in  analysis 
models  of  tl  reats  up  to  date  and  thereby  maintaining  the  currentness 
of  the  models. 

Obstacles  to  Readiness.  There  are  a  number  of  obstacles, 
such  as  the  following: 

•  Entropy  of  Analysis  Procedures  and  Models.  Procedures 
and  models  tend  to  fall  out  of  date.  For  example,  a 
working  threat  model  involving  possible  hostilities 
between  the  U.S.  and  another  country  will  tend  to  go 
out  of  date  because  of  evolutions  in  weapon  systems. 
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new  political  postures,  economic  changes,  etc.  The 
basic  antidote  is  the  process  of  analytic  maintenance 
whereby  new  information  is  used  to  update  the  given 
model . 

•  Difficulty  in  Implementing  Analysis.  A  serious 
obstacle  to  readiness  can  arise  when  analytic  tools 
and  procedures  —  including  those  up  to  date  —  cannot 
quickly  and  efficiently  be  employed  by  analysts.  For 
example,  quite  realistic  and  useful  threat  scenarios 
may  be  developed  spanning  a  very  reasonable  range  of 
possibilities,  yet  be  documented  in  forms  not  easily 
used  by  analysts  during  time-critical  periods. 
Computer-based  support  systems  which  include  displays 
of  models  might  greatly  facilitate  the  ease  of  employ¬ 
ment  of  the  models  and  techniques.  However,  the 
accessibility  of  the  computer-stored  data  then  becomes 
a  readiness  concern. 

•  Unavailability  of  Experienced  Analysts.  An  obvious 
obstacle  to  readiness  is  lack  of  training  and  experi¬ 
ence  applicable  to  different  problem  areas. 

•  Analysts'  Resistance  to  Analytic  Procedure  and 
Technique .  Analysts  may  become  ill-disposed  to 
employ  certain  procedures  and  techniques  of  analysis. 
Problems  in  psychological  readiness  can  arise  from  a 
variety  of  causes  and  are  discussed  ixi  succeeding 
portions  of  the  analysis,  particularly  in  the  context 
of  incentives . 

Typical  questions  which  center  on  the  issue  of  readiness 
are:  Is  there  a  backlog  of  input  data  to  filter  through  given  models? 
How  long  does  it  take  to  retrieve  various  portions  of  memory  stored 
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in  given  computers?  How  long  does  it  take  to  compare  past  data  to 
present  by  an  analyst?  How  comprehensively  can  a  given  number  of 
analysts  worry  different  sides  of  a  problem  within  a  limited  time 
frame? 


Preliminary  Concepts  of  Readiness  Measures.  Clearlj  there 
are  several  dimensions  to  readiness  within  the  warning  analysis 
problem,  such  as:  analyst  readiness,  system  readiness ,  and  supporting 
technology  readiness .  Analysts  must  have  experience  and  training  to 
assure  analytic  readiness.  The  system  of  analysis  itself  —  the 
models,  techniques,  procedures  —  may  be  more  or  less  ready.  Further, 
the  operations  the  analyst  performs  in  recalling  information, 
displaying  information,  matching  disparate  data  sets,  etc.,  involve 
readiness . 


Readiness  as  a  state,  a  condition,  is  ultimately  the  scate 
of  the  entire  analysis  system.  Readiness  must  be  reflected  in  a  set 
of  values  on  some  scale  which  the  key  variables  of  that  system  have 
at  any  given  instant.  Discussion  of  information-related  variables 
will  allow  us  to  explore  further  and  define  detailed  readiness 
measures . 


Maintenance 


The  third  major  measure  is  analytic  maintenance.  Mainte¬ 
nance  is  an  activity;  it  is  a  measure  of  the  sustaining  of  analytic 
procedures,  the  keeping  current  of  the  analytic  memories.  Mainte¬ 
nance  is,  in  short,  partly  accomplished  through  the  act  of  analysis 
itself. 


activity: 


It  does  not  matter  that  we  are  talking  about  a  cognitive 
when  analysts  update  models,  refine  procedures,  review 


50 

IWAMS 


input  data,  and  generally  perform  analysis,  they  are  maintaining 
readiness,  readiness  of  their  own  capabilities  and  readiness  of  the 
supporting  methodological  and  technological  systems.  This  in  turn 
influences  effectiveness. 

Obstacles  to  Maintenance.  Some  examples  are  as  follows: 

•  Failure  to  route  input  data  through  analytic 
procedures,  techniques,  and  models:  When  input  data 
is  not  routed  through  the  analytic  process,  entropy 
is  risked.  Maintenance  arises  out  of  the  connection 
of  daca  throughput  to  the  cognitive  functions  of 
analysis.  If  the  analyst  does  not  periodically  run 
the  data  through  the  system,  then  the  system  must 
lack  maintenance. 

•  Failure  to  checkout  adequately  technical  support 
systems :  This  classic  aspect  of  maintenance  refers  to 
preventative  maintenance  of  the  hardware  and  software 
support  systems.  A  display  system  that  frequently  is 
down  for  repair,  an  unreliable  computer-based  system, 
and  a  software  package  not  fully  debugged  are  obvious 
examples . 

Preliminary  Concepts  of  Maintenance  Measures .  Typical 
maintenance  questions  include*  What  is  the  periodicity  with  which 
new  input  data  has  been  run  through  the  system  of  analysis?  How 
much  preventative  maintenance  has  been  done,  and  when,  with  respect 
to  any  given  support  systems  within  the  I&W  center? 


As  the  diagram  below  shows,  we  have  now  completed  a  general 
discussion  of  the  four  major  measures  of  analysis  performance. 
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Key  Information-Related  Variables 


To  derive  real  and  estimated  effectiveness,  readiness  and 
maintenance,  key  information- related  variables  must  be  monitored. 

The  variables  should  have  certain  characteristics.  They  should  be 
interrelated  within  the  overall  process  of  analysis;  they  should  be 
recognizable  (monitorable)  and  intelligible  by  observers  such  as 
managers  and  analysts  choosing  to  attend  to  the  system;  they  must  be 
variables  which  change  over  time;  they  must  be  sensitive  and  signi¬ 
ficant  for  diagnostic  purposes;  and  their  change  must  be  measurable. 

Three  information-related  variables  have  been  developed  as 
fundamental  to  deriving  the  measurements: 

•  Data  backlog 

•  Data  relationality 

•  Data  signification 
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In  the  discussion  which  follows,  each  of  the  variables 
will  be  defined,  explained  as  to  choice,  and  exemplified.  Specific 
connections  between  the  variables  and  the  measures  will  then  be 
specified.  The  ultimate  intent  is  to  show  how  the  key  variables 
combine  with  the  measures  to  produce  a  meangingful,  tightly  coherent 
system  of  linked,  interrelated  measures. 

We  will  also  explore  certain  important  subvariables  which 
could  be  examined  on  the  basis  of  empirical  data  generated  by  moni¬ 
toring  the  three  key  information-related  variables.  That  discussion 
will  be  followed  by  a  consideration  of  the  best  uses  of  the  data; 
primarily  focused  on  the  problems  in  warning  analysis  that  might  best 
be  approached  with  various  kinds  of  data  generated  by  the  measurement 
system.  Finally,  certain  conditional  variables  related  more  to  the 
information  input  aspect  than  throughput  will  be  discussed. 

Backlog 

The  classic  variable  of  backlog  here  refers  essentially 
to  identifying  information  anywhere  in  the  analytic  process  which  is 
in  a  waiting  position,  which  has  yet  to  be  fully  analyzed  by  warning 
analysts;  which  has,  in  short,  not  completed  throughput.  We  must 
think  of  the  backlog  in  relation  to  all  points  —  stages,  procedures, 
steps  —  in  the  total  model  of  the  analytic  process. 

It  is  desirable  to  measure  a  number  of  backlog  values,  such 
as:  the  size  of  the  backlog,  the  nature  of  the  backlog  (e.g.,  the 
types  of  data  which  comprise  it),  the  time  duration  of  the  backlog, 
and  such  dynamic  measurements  as  the  rate  of  decrease  or  increase  of 
backlog,  the  acceleration  values  within  the  rates  of  increase  and 
decrease,  and  other  measures.  These  of  course  could  be  related  to 
other  variables  such  as  number  of  analysts,  technical  support,  etc. 
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Backlog  is  a  fundamental  variable  within  the  system,  for 
a  great  portion  of  the  entire  approach  is  founded  on  the  need  and 
desirability  of  throughput.  Readiness,  maintenance  and  effective¬ 
ness  have  all  been  related  in  different  fashions  to  the  process  of 
throughput..  (In  their  extended  visits  to  I&W  centers,  the  authors 
hive  become  fully  aware  of  the  relatively  large  volumes  of  input 
data  typically  on  hand  that  must  be  processed  and  analyzed.  We 
will  discuss  the  "volume"  problem  in  a  later  section,  indicating 
for  now  that  we  do  not  view  it  as  an  insurmountable  obstacle  to 
achieving  necessary  backlog  determination. ) 


Relationality 

The  relationality  variable  involves  the  extent  to  which 
throughput  data  is  related  to  the  full  system  of  analytic  stages, 
steps,  procedures,  and  methods  in  the  overall  methodology.  Relation¬ 
ality  embodies  the  following  question:  was  the  data  analyzed,  or  is 
the  data  now  being  analyzed,  through  all  or  some  (and  if  so,  which) 
of  the  procedural  routines,  models,  and  projection  techniques  which 
comprise  the  analytic  system  of  warning?  The  analytic  system  is  the 
analyst's  best  (if  inevitably  imperfect)  means  of  attacking  the 
epistemological  an!  cognitive  problems  in  warning.  The  extent  to 
which  the  analyst  appropriately  follows  that  system  bears  directly 
on,  first,  estimate!  effectiveness;  second,  real  effectiveness  (to 
the  extent  that  the  system  has  been  demonstrated  as  empirically 
effective);  third,  maintenance,  since  the  act  of  analysis  itself 
comprises  a  large  portion  of  analytic  maintenance;  and  finally, 
readiness,  since  readiness  is  the  state  or  condition  of  certain 
analytic  abilities.  !Tn  short,  relationality,  like  backlog,  is 
fundamental  to  the  basic  measures. 
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Furthermore,  relationality  and  backlog  have  an  important 
complementarity:  reduction  of  data  backlog  should  occur  through 
relationality  to  the  procedures  and  methods  within  the  warning  system. 
The  two  variables  are  interrelated;  together  they  indicate  far  more 
about  warning  analysis  than  taken  separately. 


Signification 

The  most  important  of  the  three  variables,  signification, 
is  also  the  most  difficult,  the  most  experimental.  Signification 
involves  the  meaning  assigned  to  data  by  analysts.  Signification 
is  captured  by  recording  changes  at  certain  nodes  in  the  analytic 
process;  for  example,  changes  in  variables  in  threat  models  and 
changes  to  the  structure  of  threat  models  and  the  reason(s)  for  the 
change.  As  discussed  above,  meaning,  taken  as  constituting  the 
significance  of  information  to  a  system  which  processes  it,  incurs 
a  change  in  that  information  system's  processes  elicited  by  the 
information  input,  the  change  often  resulting  from  associations 
made  to  the  data  based  on  previous  experience  with  it.  Change  car 
occur  immediately  or  later.  Given  that  we  are  viewing  analysis  in 
systemic  terms  —  a  process  with  stages,  steps,  procedures  —  the 
monitoring  of  meaning  through  changes  expressed  at  nodes  in  our 
functional  model  becomes  a  natural  strategy. 

The  warning  analyst  essentially  attempts  to  determine  the 
meaning  of  input  information  in  terms  of  the  degree  (if  any)  to  which 
it  may  signify  the  emergence  of  a  threatening  situation.  The  Soviets 
distinguish  this  act  as  a  form  of  decisionmaking,  referring  to  it  as 
an  information  decision.  (The  analyst  is  also  interested  in 
recognizing  nonthreatening  situations  and  therefore  assigns  meaning 
to  the  data  in  any  event.)  The  crucial  point  is  this:  if  we  wish 
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to  explore  real  effectiveness,  we  must  be  able  to  perform  post 
mortems  which  essentially  are  reconstructions  of  past  analytic 
states  and  perspectives,  which  leads  us  directly  to  the  need  for  a 
selective  history  of  meaning. 

The  question  invariably  arises,  Why  take  on  a  problem  as 
complex  and  subtle  as  the  problem  of  measuring  meaning?  We  know  that 
objective  quantification  is  not  possible  at  the  present  time.  We 
know  that  contemporary  models  of  human  cognition  are  by  no  means 
complete,  and  that  we  can  make  no  pretense  of  fully  understanding 
and  hence  capturing  the  detailed  (and  mercurial)  processes  by  which 
meaning  occurs  in  a  living  system.  What,  then,  are  the  propects  for 
achieving  useful  results?  What  are  the  circumscribed  goals?  To 
answer  these  questions,  we  must  re-explore  a  central  issue  in  warning. 

The  Problem  of  Real  Effectiveness.  Real  effectiveness  is, 
of  course,  the  ultimate  frame  of  reference  for  warning  analysis.  Tts 
measurement  vehicle  is  the  post  mortem.  Post  mortems  can  be  formal, 
informal,  limited  to  "failures,"  conducted  in  all  seasons,  etc.  But 
stated  simply,  real  effectiveness  involves  these  questions:  Looking 
back,  how  well  did  we  do?  And  why? 

Besides  the  political  and  technical  difficulties,  the 
post  mortem  process  is  beset  with  evidentiary  difficulties  arising 
from  cognitive  and  other  constraints.  These  latter  kinds  of  diffi¬ 
culties  in  conducting  hindsight  analysis  will  always  (and  most 
fundamentally)  complicate  the  determination  of  real  effectiveness. 

A  useful  starting  point  is  this:  analytic  memory  is 
crucial  for  post  mortems  and  the  determination  of  real  effectiveness. 
But  there  are  serious  constraints  involved  with  analytic  memory.  The 
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problems  may  usefully  be  expressed  in  terms  of  cognitive  biases  such 
as  discussed  earlier.  Richards  J.  Heuer,  Jr.,  has  recently  reviewed 
and  constructively  interpreted  the  research  literature  in  this  area. 
He  concludes  that: 

®  Analysts  tend  to  overestimate  the  worth  and  accuracy 
of  their  past  analytic  judgments. 

•  The  consumer  of  the  information  (in  this  case  the  user 
of  intelligence  information)  tends  to  underestimate 
the  value  of  that  intelligence  to  him. 

•  The  manager  conducting  a  post  mortem  tends  to  conclude 
that  events  were  more  readily  foreseeable  (or  less  so) 
than  was  in  fact  the  case. 

•  In  short,  we  seem  to  have  a  systematic  tendency  toward 
faulty  memory  of  our  past  estimates . 

The  explanation  for  these  problems  comes  largely  out  of 
experimental  research.  As  Heuer  has  told  us,  there  art.  several  key 
points : 

•  It  appears  that  additional  information  available  for 
hindsight  analysis  actually  changes  our  perceptions  of 
a  situation  so  naturally  and  so  immediately  that  we 
are  largely  unaware  of  the  change. 

•  The  new  input  data  is  immediately  (and  probably  uncon¬ 
sciously)  assimilated  into  our  prior  knowledge. 
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•  If  the  new  data  adds  significantly  to  our  knowledge  — 
tells  us  a  situation  outcome  —  our  mental  images  are 
reconstructed. 

•  Apparently  it  is  then  virtually  impossible  for  us  to 
reconstruct  cur  prior  mental  set. 

•  Thus,  we  may  recall  but  not  reconstruct. 

Ironically  the  human  being  at  this  stage  of  his  mental 
development  has  a  memory,  a  data  base,  which  is  perturbed,  altered 
and,  in  fact,  progressively  lost  in  important  ways;  and  lost,  more¬ 
over,  somewhere  within  the  vital  and  necessary  act  of  taking  in  new 
data,  throughputting  the  data  within  the  human  congitive  process, 
and  changing  perspective  on  the  external  world.  Thus  in  the  act  of 
learning  we  also  forget. 

In  principle,  there  is  a  remedy:  extrasomatic  memory.  In 
discussing  the  development  and  outlook  for  human  intelligence, 

Carl  Sagan  and  others  have  stressed  the  importance  of  information 
stored  outside  the  body  —  the  early  invention  of  libraries  of  the 
written  word,  and  the  future  prospects  of  more  advanced,  inter¬ 
actively  dynamic,  forms  of  extrasomatic  information  bases  which  use 
computer  technology.  In  some  commentaries,  there  is  a  sense  of 
urgency,  the  belief  that  the  world  is  changing  more  quickly  than 
ever  before,  and  that  to  perceive  the  changes  and  react  in  ways  that 
foster  survival,  learning  systems  supported  by  advanced  information 
technology  are  crucial. 

The  image  of  the  world  changing  ever  more  quickly  — 
with  the  change  substantially  of  our  own  making  —  and  the  change 
increasingly  taxing  our  perceptual,  cognitive  and  hence  survival 
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powers,  obviously  and  fundamentally  bears  on  the  warning  problem. 

For  the  present,  it  is  instructive  to  cite  only  one  of  the  many 
problems  in  the  current  dilemma.  This  is  the  problem  of  the 
increasingly  stringent  requirements  to  monitor  significant 
change,  especially  threatening  change,  in  areas  physically  remote 
from  the  immediate  environment  of  the  perceiver.  The  "remote" 
quality  obviously  carries  over  into  political,  cultural,  economic, 
and  other  dimensions,  thereby  complicating  recognition  by  the 
watcher  on  all  levels  of  analysis.  The  need  to  watch  events  abroad 
is  not,  of  course,  new;  the  problem  is  that  the  reaction  times  and 
other  constraints  on  the  watch  are  growing  more  severe.  Yet  rhe 
evolution  of  human  intelligence  may  be  at  a  stage  in  which  we  are 
still  operating  in  part  as  if  the  threat  from  extinct  predators  is 
more  real  than  the  complex  international/intercontinental  threats. 

Can  there  be  any  doubt  that  unaided  human  perception,  analysis  and 
decision-making  are  inadequate  to  the  challenge?  It  seems  obvious 
that  information  technology  is  essential  to  survival.  Certainly  it 
is  our  view  that  information  technology  is  now,  and  increasingly  will 
be,  vital  to  the  warning  analyst. 

Focusing  on  the  warning  issue  —  the  problem  of  real  effec¬ 
tiveness,  post  mortems  and  learning  —  let  us  suppose  that  as  analysis 
progressed  in  an  analytic  community  we  could  succeed  in  steadily, 
continuously  capturing  signification  values  (as  well  as  backlog  and 
relationality  values)  and  storing  them  in  an  extrasomatic  memory, 
where  computer  programs  would  perform  search-recall-reduction- 
re^onstruction  functions  and  display  the  results  in  various  forms, 
including  boiled  down  sequential  or  narrative  accounts  of  the 
history  of  the  development  of  given  analytic  rationales  —  the  input 
data,  its  impact,  the  assumptions,  and  the  logic  (or  methodologies). 

In  theory,  there  would  be  several  benefits.  The  recorded 
memory  of  meaning  would  be  preserved  safe  from  the  perturbations  in 
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the  human  memory.  Further,  the  basis  would  be  laid  for  an  improved 
instititionalized  procedure  for  reliably  obtaining  a  permanent, 
accurate  record  of  certain  key  analytic  constructs  —  constructs 
developed  by  changing  communities  and  groups  of  analysts  over  time  — 
which  we  would  need  for  post  mortems  and  the  measurement  of  real 
effectiveness  as  well  as  for  operations  research  and  the  definition 
of  R&D  requirements.  Among  other  things,  the  data  base  developed 
would  be  beneficial  to  research  and  innovation  in  methodology, 
analytic  incentives,  and  in  other  critical  areas  in  which  the  basic 
constraints  on  analytic  effectiveness  must  be  challenged  if  there  is 
to  be  progress.  Finally,  we  would  be  creating  the  basis  for  contin¬ 
uous  post  mortems  internal  to  the  analytic  community  and  conducted 
by  analysts  and  managers  at  all  levels.  Handled  properly,  this  capa¬ 
bility  would  hold  some  advantages  over  (if  not  necessarily  remove 
the  need  for)  post  mortems  done  essentially  by  "outsiders"  such  as 
appointed  committees  who  explore  an  intelligence  failure.  Of  course, 
the  computer-based  technology  supporting  the  extrasomatic  memory  must 
be  designed  such  as  to  eliminate  unacceptable  drain  on  analysts' 
time.  Obviously  behavioral  ironies  created  by  such  a  system  among 
analysts  and  managers  is  also  a  serious  potential  problem  to  weigh 
carefully. 


Levels  of  Signification.  Two  related  questions  now  arise: 

To  what  extent  can  we  measure  signification?  To  what  extent  should 
we  measure  it?  The  answers  to  both  involve  our  knowledge  of  human 
cognicive  processes.  As  we  know,  brain  research  has  led  to  mappings 
and  models  of  the  brain  to  varying  levels  and  in  various  dimensions  — 
chemical,  electrical,  neurological,  functional,  etc.  Mental  processes 
have  been  classified  (intuition,  rationality)  and  associated  with 
cognitive  activities  (counting,  writing,  language  comprehension, 
nonverbal  ideation,  etc.)  More  elaborately  defined  models  are  being 
developed  as  new  research  leads  to  further  understanding.  We  have 
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known  for  some  time,  however,  that  the  operations  of  the  mind  on  data 
are  enormously  complex.  There  are,  for  example,  millions  of  light 
receptors  associated  with  the  retina  of  the  human  eye  which  receives 
visual  input.  There  are  perhaps  a  million  ganglion  cells  which 
receive  signals  from  the  receptors  in  the  retina  for  processing  prior 
to  the  throughput  of  the  data  to  a  section  of  the  cerebral  cortex,  to 
be  followed  by  other  micro  operations  cf  great  complexity.  Thus 
when  we  refer  to  the  information  subsystem  within  a  human  system 
(input  transducer,  decoder,  encoder,  associative  functions,  memory, 
etc.)  we  are  considering  a  process  which  in  its  fundamental  levels 
involves  millions  of  parts  and  processes. 

The  need  therefore  arises  for  a  tractable  yet  viable  model 
of  human  cognitive  activity,  constructed  at  a  level  of  detail  suitable 
for  present  purposes.  Too  simplistic  a  model  will  not  yield  enough 
variables  for  productive  strategies  of  system  development  to  emerge. 
Too  intricate  a  model  —  a  detailed,  neurologically-based  model,  for 
example  —  would  obscure  the  focus  on  key  warning  analysis  problems. 

The  attempt  to  develop  a  functional  model  of  the  cognitive 
steps  in  warning  analysis  (Section  2)  is  thus  judged  an  appropriate 
general  strategy.  However,  we  must  search  for  a  real  and  usable 
analog  in  human  cognition  to  the  potential  signification-measurement 
points  in  the  model.  Recall,  we  have  adopted  the  following  strategy 
for  measuring  signification:  we  will  create  a  set  of  models  of 
threat  situations;  projection  models;  and  other  nodes  in  the  analytic 
process  which  are  (a)  subject  to  change  by  an  analyst  as  a  function 
of  the  significance  of  input  data,  (b)  visible  to  monitoring,  and 
(c)  likely  to  reflect  important  changes.  Recorded  changes  would 
comprise  the  memory,  which  should  be  extrasomatic.  (We  will  exem¬ 
plify  and  discuss  the  design  features  of  such  nodes  in  detail  in 
Section  4. ) 
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We  know  that  we  cannot  claim  that  such  an  approach  will 
capture  all  aspects  of  signification.  This  is  because  of  factors 
such  as  (a)  we  do  not  now  understand  mental  processes  thoroughly 
enough,  (b)  changes  in  the  analyst's  mind  caused  by  input  data  and 
reflecting  meaning  are  "subjective,"  (c)  the  changes  are  virtually 
continuous,  (d)  they  are  perhaps  not  fully  conscious,  and  (e)  they 
are  always  subject  to  loss  in  the  sense  discussed  above.  Thus 
we  will  have  an  incomplete  record  of  signification.  For  example, 
even  within  a  system  in  which  analysts  indicated  and  recorded  signi¬ 
fication  according  to  certain  criteria  and  within  a  schedule,  there 
would  still  be  some  loss,  especially  in  terms  of  the  items  listed 
immediately  above. 

Can  we,  though,  have  a  realistic  and  useful  if  limited 
record?  One  that  serves  both  a  valuable  present  purpose  and  becomes 
the  first-generation  memory  in  an  evolving  series  of  increasingly 
refined  and  productive  memories?  In  the  authors'  judgment  the  answer 
is  that  we  can. 

First,  there  is  an  apparent  solution  to  the  problem  of 
finding  a  functional  analog  in  the  human  mind  to  the  extrasomatic 
system  of  measuring  signification.  Specifically,  there  is  something 
of  an  analog  between  an  extrasomatic  model  of  a  threat  situation 
stored  in  a  computer  and  recallable  in  display  graphics  form  and 
whose  nodes  of  activity  can  be  changed  by  an  analyst  through  the 
new  meaning  conveyed  by  input  data  (signification),  and  a  configura¬ 
tion  carried  within  the  human  brain  which  cognitive  psychologists 
call  schemata  or  plans  (sometimes  described  as  a  "template").  One 
technical  definition  of  a  schemata  is: 

....  a  configuration  within  the  brain,  either  inborn  or 
learned,  against  which  the  input  of  the  nerve  cells  is 
compared.  The  matching  of  the  real  and  expected  patterns 
can  have  one  or  the  other  of  several  effects.  The  schema 
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can  contribute  to  a  person's  mental  'set,'  the  screening 
out  of  certain  details  in  favor  of  others,  so  that  the 
conscious  mind  perceives  a  certain  part  of  the  environment 
more  vividly  than  others  and  is  likely  to  favor  one  kind  of 
decision  over  another.  It  can  fill  in  details  that  are 
missing  from  the  actual  sensory  input  and  create  a  pattern 
in  the  mind  that  is  not  entirely  present  in  reality.  In 
this  way  the  gestalt  of  objects  --  the  impression  they 
give  of  being  a  square,  a  face,  a  tree,  or  whatever  —  is 
aided  by  the  taxonomic  power  of  the  schemata. 

The  schemata  plays  a  decisive  role  in  human  cognition, 
particularly  with  respect  to  such  analytic  functions  as  interpretation, 
recognition,  and  extrapolation,  and  hence  seems  an  appropriate  level 
within  human  cognitive  operations  with  which  to  draw  a  correspondence 
in  designing  a  functioning  extrasomatic  memory  of  signification. 

From  the  imputed  correspondence  between  the  schema ua  and  the  extraso¬ 
matic  memory  of  meaning,  configured  as  it  is  in  models  which  are 
themselves  a  kind  of  schemata,  we  develop  a  possible  basis  for  a 
realistic  and  useful  record  of  analytic  meaning. 

One  major  prospect  is  that  the  imputed  correspondence  offers 
hope  not  only  of  at  least  some  initial  realism  in  the  structual  design 
of  the  extrasomatic  memory,  and  hope  not  only  that  the  analyst  has  no 
fundamental  behavioral  aversion  to  the  structure  (that  the  "human 
factors"  issue  is  not  overriding):  it  also  offers  hope  that  the  extra¬ 
somatic  memory  will  quite  naturally  improve  over  time,  be  refined, 
made  increasingly  more  compatible  for  human  usage,  and  hence  become 
more  powerfully  reflective  of  signification,  since  one  might  reasonably 
expect  that  continued  interaction  between  the  analyst  and  the  extraso¬ 
matic  memory  (in  practical  terms,  of  course,  a  man-machine  interaction) 
will  naturally  lead  to  a  convergence,  a  humanizing  of  the  extrasomatic 
memory . 

There  remains,  however,  the  question  of  incomplete  signi¬ 
fication.  Ordinarily  we  will  not  be  able  to  record  signification 
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operations  in  the  brain  on  a  microsecond-to-microsecond  basis . 

Ideally  should  we  monitor  signification  at  such  levels?  Since  we 
cannot,  what  is  the  effect  on  the  utility  of  the  signification 
measurement?  In  our  judgment  we  are  very  probably  not  losing  enough 
to  impede  important  measurement.  A  grasp  of  the  microprocessns  of 
signification  may  not  lead  to  rapid  improvements.  Assuming  t  iat  we 
someday  understand  them,  these  microprocesses  are  likely  to  reveal 
considerably  more  detail  about  an  important  fact  of  which  we  already 
are  only  too  well  aware:  human  fallibility.  We  can  now  estimate 
fairly  accurately  storage  and  processing  capacities  of  the  mind, 
without  knowing  fully  its  cognitive  microprocesses,  we  can  also 
recognize  beyond  any  doubt  that  the  progress  of  the  human  mind  in 
understanding  reality  has  been  painfully  slow.  We  do  not  need  to 
know  more  at  this  point  in  order  to  pursue  signification  usefully. 

There  are  other  aspects,  such  as  the  nature  and  role  of  the 
collective  extrasomatic  memory,  which  we  will  discuss  below.  But  we 
may  interrupt  the  discussion  of  signification  for  now  with  the  obser¬ 
vation  that  obviously  it  will  be  necessary  to  collect  empirical  data 
over  time  to  assess  the  utility  of  the  specific  signification  measure¬ 
ment  points  chosen  in  the  design.  In  later  sections  dealing  with  the 
IWAMS  design,  key  nodes  are  identified  in  the  analytic  process  believed 
important  in  capturing  signification,  and  the  design  of  these  nodes 
is  exemplified  in  some  detail.  A  simple  example  is  certain  types  of 
information  filters  existing  in  threat  models  by  which  input  informa¬ 
tion  may  be  associated  with  preestablished  meaning.  In  this  type  of 
signification  node,  input  data  may  be  clustered  around  certain  nodes 
which  model  events  hypothesized  likely  given  the  activity  of  certain 
indicators . 


64 

IWAMS 


The  Measures  and  Variables  as  a  System 


We  have  considered  the  measures  and  variables  individually 
but  their  full  definition  resides  in  their  linkage,  interrelatedness, 
and  interdependence.  We  must  now  explore  how  the  basic  analysis 
measures  —  maintenance,  readiness,  estimated  effectiveness  and  real 
effectiveness  —  connect  to  the  key  variables  of  backlog,  relation- 
ality,  and  signification;  and  how  this  creates  linkage  among  the 
measures,  thereby  meeting  one  of  the  IWAMS  design  goals. 

Table  4  is  a  matrix  summarizing  these  interrelationships. 

Commentary  on  Table  4.  As  shown  in  the  table,  maintenance 
is  associated  with  four  variables:  standard  preventative  maintenance 
(PM),  backlog  (BL),  relationality  (R^),  and  signification  (ST). 


The  maintenance  measure  depends,  firsr,  on  monitoring  the 
time  that  support  systems  are  used  versus  the  vendor  repair/downtime 
schedule,  which  provides  preventative  maintenance  measures.  Second, 
it  requires  measuring  backlog,  derived  by  determining  what  data  in 
given  areas  (e.g.,  the  North  Korean  problem)  has  not  been  analyzed 
within  given  analytic  stages . 


Third,  maintenance  entails  measuring  relationality.  In 
relationality  one  factor  taken  into  account  is  the  time  spent  by 
analysts  in  relating  given  data  to  given  models,  in  following  proce¬ 
dures,  and  in  employing  other  analytic  techniques.  For  a  number  of 
reasons  managers  and  analysts  possibly  might  wish  to  monitor  and 
compare  such  values.  One  reason  can  be  suggested  by  drawing  an  analogy 
to  the  analysis  of  intelligence  imagery.  In  some  analysis  of  aerial 
photography,  the  objective  may  not  be  to  do  detailed,  painstaking 
review  of  various  photographed  scenes,  especially  if  the  photography 
does  not  give  adequate  resolution  for  extremely  fine  analysis .  The 
purpose  of  analyzing  such  low  resolution  imagery  might  be  to  enable 
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Table  4.  Interrelationships  Among  Measures  and  Variables 


Analysis 

Measure 

Variable 

Data  To  Be  Monitored 

Calculation 

Maintenance 
(M)  (an 
activity) 

Preventative 

maintenance 

(PM) 

Time  used  versus  vendor 
schedule 

PM  =  I  ts 

Backlog  (BL) 

Data  on  given  geographic 
areas  (or  other  problem 
categories)  not  analyzed 
at  given  stages  in  analy¬ 
tic  process 

BL  =  I  area  BL 

Relationality 

(V 

' 

Analytic  models/ 
procedural  routines 
involved  versus  total 
available  data. 

#  models 

RT"ZTi 

Time  spent  on  each  model 
(T.) 

Signification 

<sT) 

Changes  to  variables  at 
analytic  nodes. 

ST  =  1  Si  ' 

Account  taken  of  analyst 
skill  (Aj ) 

Readiness  (R) 
(a  state) 

NA 

HA 

Scaling  based  on  PM,  BL 
RT  and  ST  measures 

(ideal  M  versus  real  M) 

Ob  j  ective/  sub  j  ec  tive 
reference  points  on 
scales 

Estimated 
Effectiveness 
(EE)  (a 
result  esti¬ 
mated 
a  priori) 

Forecast 
probabilities 
as  determined 
by  analyst 

Forecasts  in  projection 
stage  of  analysis 

EE  =  Z  R  +  forecast 
probabilities 
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Table  4.  —  Continued. 


Real 

Forecast 

Forecasts  in  projection 

RE  =  I  EE  +  €  (PT) 

Effectiveness 

probabilities 

stage  of  analysis.  Subse- 

(PE)  (a 

as  determined 

quent  real  events  in 

result  sought 
to  be  deter¬ 
mined  in  post 
mortem) 

by  analyst 

projected  period  [via  post 
mortem  (PT)1 

I 
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search  of  areas  to  determine  possible  targets  which  subsequently 
should  be  monitored  in  higher  resolution  imagery.  The  analytic  time 
spent  on  a  single  frame  of  high  quality  imagery  of  a  significant 
target  ordinarily  would  be  greater  than  on  a  lower  resolution  image 
of  the  same  area.  Similarly,  different  stages  in  the  progressively 
more  difficult  challenges  over  the  sequence  of  warning  analysis  from 
monitoring  to  projection  should  result  in  variations  in  times 
reflected  in  relationality.  With  enough  data,  basic  trends  anil 
characteristic  time  patterns  might,  within  certain  constraints,  begin 
to  emerge  as  a  set  of  analytic  norms.  Such  data  might  be  very  impor¬ 
tant  for  diagnostics:  for  example,  in  operations  analysis  and  research 
and  development  activities  designed  to  afford  analysts  more  time  to  do 
the  more  difficult  parts  of  analysis  and  to  reduce  time  delays  in  parts 
of  the  process  which  should  be  done  more  quickly.  Another  purpose  in 
examining  comparative  times  is  to  build  in  a  safeguard  against  attempts 
to  reduce  backlogs  by  skimping  on  conducting  the  relational  aspect  of 
analysis.  Moreover,  the  time  factor  ultimately  could  more  sharply 
delineate  for  managers  dilemmas  created  by  limited  analytic  resources. 
But  obviously  caution  and  empirical  wisdom  are  needed  since  there  are 
not  necessarily  ideal  times  for  performing  various  portions  of 
analysis.  Clearly,  analysts  will  differ  in  ability,  problems  will 
differ  in  complexity,  and  data  will  differ  in  their  applicability. 

In  the  final  analysis,  the  inclusion  in  the  software  design  of  the 
capability  to  monitor  times  spent  in  relationality  and  in  reducing 
backlog  might  aid  in  the  general  effort  to  ensure  that  a  variety  of 
questions  of  a  diagnostic  nature,  many  not  now  fully  anticipated, 
could  be  answered  in  the  future. 

Signification  values  are  also  determined  for  the  maintenance 
measure.  Note  that  the  calculation  of  signification  includes  a  value, 

A j ,  which  refers  to  the  estimated  skill  of  the  analyst(s).  This  is 
an  experimental  factor.  "Skill"  could  be  derived  from  experience, 
training,  demonstrated  interpretive  success,  etc.  Empirical  data 
and  a  period  of  testing  the  practicality  and  utility  of  this  factor 
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are  required.  It  would  prove  interesting  in  later  research  to 
attempt  to  determine  correlations  between  different  analyst  profiles 
and  signification.  In  later  discussions  of  the  operation  of  IWAMS, 
we  will  explore  the  approach  of  developing  IWAMS  specialists  who 
largely  (but  not  exclusively)  operate  the  system.  The  Aj  factor 
would  be  of  interest  here. 

Readiness ,  an  analytic  state,  is  expressed  in  the  form  of 
scaling  based  on  PM,  BL,  RT  and  ST-  Readiness  is  basically  derived 
by  comparing  ideal  maintenance  to  real  maintenance.  Note  also  that 
under  the  calculation  of  readiness  there  is  mention  of  objective 
and  subjective  reference  points  on  the  readiness  scales.  Managers 
of  analytic  centers  must  allocate  limited  analysis  resources  to  a 
wide  range  of  monitoring  responsibilities.  Managers  may  assign 
different  goals  of  readiness  to  different  problem  areas  and  different 
cadres  of  analysts. 

Under  estimated  effectiveness,  ir  is  seen  that  the  analyst 
will  determine  projection  probabilities  (this  is  illustrated  in 
considerable  detail  in  succeeding  sections)  and  that  the  projections 
will  have  readiness  factored  in  as  a  means  of  determining  estimated 
effectiveness.  In  sum,  current  readiness  should  qualify  projection 
probabilities  and  produce  estimated  effectiveness. 

Real  effectiveness,  as  discussed  above,  is  ultimately 
determined  through  post  mortems .  In  subsequent  sections,  the  problem 
of  measuring  real  effectiveness  is  considered  in  detail.  It  is 
necessary  to  consider  analytic  procedures,  forms,  and  methods  in  more 
substance  before  appropriately  addressing  real  effectiveness. 

The  diagram  shown  below  indicates  our  present  level  of 
conceptualization  of  the  full  system  of  analysis  performance  measures 
and  the  relationship  to  the  key  variables 
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RELATIOtMALITY  MEASURE  OF  EXTENT  TO  WHICH  INPUT  DATA  IS  ANALYZED 
BY  RELATING  IT  TO  SYSTEM  OF  ANALYTIC  PROCEDURES. 
MODELS,  METHODOLOGIES 

SIGNIFICATION  MEANING  ASSIGNED  TO  DATA  BY  ANALYSTS.  CAPTURED  BY 
RECORDING  CHANGES  AT  CERTAIN  NODES  IN  ANALYTIC 
PROCESSfe*..  CHANGES  IN  VARIABLES  IN  THREAT  MODELS! 
AND  ASSOCIATED  RATIONALE 


The  definitions  that  now  appear  in  the  wedges  within  the  diagram 
under  the  four  basic  measures  are  explicitly  related  to  the  key 
variables.  In  addition,  the  four  arrows  positioned  at  different 
points  along  the  circumference  of  the  circle  are  meant  to  indicate 
the  linkage  among  and  across  the  measures  and  variables. 


Table  5  makes  more  exp] icit  the  linkage  between  mainten- 
ance  and  readiness,  and  between  readiness  and  estimated  effectiveness 
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Table  5.  Measures  Linkage 


From 

m  ^ 

J.  O 

- - - - - — - - - "i 

Calculated  By 

Maintenance 

Readiness 

Measured  readiness  times  a  function* 
of  maintenance  activity/effort 
required  to  bring  maintenance  up  to 

100  percent  for  specific  problem 
area  or  stage  of  analysis. 

R  •  f  (M100%  -  Mcurrent) 

Readiness 

Estimated 

Measured  estimated  effectiveness 

Effectiveness 

times  a  function*  of  the  differen¬ 
tial  between  measured  readiness 
and  optimum  readiness 

*Functions  will  be  refined  on  the  basis  of  empirical  data 
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Note  that  the  link  from  maintenance  to  readiness  is  based  on  the 
concept  that  a  certain  degree  of  effort  and  activity  in  the  form  of 
maintenance  will  be  identified  that  would  be  required  to  bring 
maintenance  up  to  100  percent  for  any  analytic  area  or  stage  of 
analysis  at  any  given  time,  (Empirical  wisdom  is  again  essential 
here . )  This  perspective  would  be  important  for  the  manager  who  must 
allocate  limited  resources  in  order  to  respond  to  problems  of 
varying  and  variable  priority. 

Subvariables.  Many  perspectives  can  be  gained  by 
collecting  data  on  the  three  major  variables.  For  example,  a  number 
of  subvariables  relate  to  the  major  variables.  Examples  include: 

•  Types  of  data.  One  can  consider  the  different  cate¬ 
gories  of  information  and  how  they  are  being  analyzed 
at  the  different  analysis  stages.  We  can  consider 
product  types  of  intelligence  —  SIGINT,  COMINT, 
IMINT,  HUMINT,  etc.  We  can  additionally  consider 
content  categories  such  as  political,  economic, 
military,  etc.  In  short,  information  may  be 
categorized  by  various  criteria  and  monitored 
thusly. 

•  Percentages .  It  will  be  important  to  determine 
percentages  of  information  at  different  levels  of 
the  analytic  process.  In  turn,  percentages  can  be 
related  to  types  of  information,  or  can  be  related 
to  other  parameters. 

•  Capacity.  Such  values  as  data  capacities  in  various 
levels  of  the  system  memory,  or  data  capacities  with 
respect  to  throughput  capabilities  at  various  nodes, 
can  be  of  interest. 
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•  Time.  One  can  monitor  time  variously:  variables 
include  durations  of  certain  processes;  speeds  of 
certain  functions;  rates  of  certain  activities,  for 
example,  rates  of  backlog  reduction;  and  accelera¬ 
tions  . 

•  Forms .  The  forms  that  certain  types  of  data  tend  to 
take  and  then  lose  during  different  processes  within 
the  analytic  model  could  be  of  interest. 

•  Gaps .  Data  gaps  and  activity  gaps  in  the  process  of 
analysis  could  be  of  considerable  interest  in  terms 
of  various  measurement  problems. 

States  and  pathologies.  We  can  think  of  a  warning  analysis 
system  as  exhibiting  different  states.  Measurement  data  may  eventu¬ 
ally  allow  us  to  explore  acceptable  steady  states.  To  define  such 
states,  we  would  need  to  determine  heuristically  relationships 
between  input,  throughput,  and  output  capabilities  and  factors  such 
as  the  number  of  analysts,  the  state  of  the  technology  support,  etc. 
Such  studies  might  eventually  provide  considerable  insight  into 
technological  solutions  for  problems  in  readiness  and  effectiveness. 

Pathological  states  in  information  systems  occur  when  one 
or  more  variables  remain  for  a  significant  period  beyond  their  ranges 
of  stability;  or  when  the  costs  of  adjustment  processes  required  to 
keep  them  within  their  ranges  of  stability  are  significantly  increased; 
or  when  the  changes  from  an  efficient  state  are  caused  by  either 
malfunctioning  of  subsystems  and  components  or  unfavorable  conditions 
in  the  environment. 

Major  categories  of  causes  of  pathological  states  in  infor¬ 
mation  systems  include  the  following: 
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Excesses  of  information  input.  The  problem  of 
excessive  information  is,  of  course,  highly  germane 
to  warning  analysis.  Adjustment  procedures  on  the 
part  of  analysts  would  be  of  key  interest  to  management. 
Excesses  in  information  input  could  cause  delays,  loss 
of  focus,  and  create  inappropriate  emphasis  on  certain 
portions  of  the  analytic  process  to  the  relative 
neglect  of  others.  All  this  in  turn  relates  closely 
to  the  epistemological  and  cognitive  problems  which 
beset  analysts  in  any  event. 

Lack  of  information  input.  Lack  of  information  input 
is  also  very  relevant  to  warning  analysis.  We  1  ave 
discussed  this  problem  in  terms  of  epistemological 
and  cognitive  obstacles  to  effective  analysis;  one 
can  see  the  relationship  to  the  analytic  process  as 
a  system.  Indeed,  the  lack  of  information  could 
increase  the  risk  of  bad  information  decisions. 

Inputs  of  erroneous  or  mischievous  information.  The 
effects  on  the  warning  analytic  system  of  the  input 
of  deceptive  data,  erroneous  data,  and  data  of  rel  - 
tively  poor  credibility  obviously  could  become  ser  ous 
obstacles  to  effectiveness. 

Abnormalities  in  internal  information  processes .  r ‘.uch 
abnormalities  must  be  identified  and  corrected  in  a 
system  of  warning  analysis.  Specific  pathologies  could 
involve  the  transmission  of  information,  the  coding  of 
information  within  the  system,  the  associating  of 
information,  decisions  concerning  the  met  ning  of  infor¬ 
mation,  and  other  activities  of  information  processing 
subsystems.  Examples  could  include  semantic  problems; 
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errors  in  routing  of  information;  and  information 
that  is  poorly  stored  in  memory,  forgotten,  not 
readily  retrieved. 

In  relation  to  such  pathologies,  there  is  a  set  of  classic 
hypotheses  about  the  likely  behavior  of  information-related  systems 
under  varying  conditions;  for  example,  when  changes  are  introduced  in 
volume,  type  of  information,  stress,  etc.  The  authors  are  confident 
that  the  measures  data  to  be  generated  by  the  IWAMS  system  would 
facilitate  useful  research  into  such  hypotheses. 

We  shall  return  to  the  discussion  of  measures  and  pathol¬ 
ogies  later.  For  now  we  may  conclude  the  present  discussion  by 
making  some  preliminary  comments  on  the  uses  of  the  kinds  of  measure¬ 
ments  we  have  been  exploring. 

Preliminary  Comments  on  the  Uses  of  Measurement  Data 

The  previous  sections  discussed  the  nature  of  the  measures 
developed  for  IWAMS.  We  should  consider  also  the  limitations  on  the 
measures  data  and  explore  appropriate  uses  of  the  data.  Of  course, 
full  answers  to  both  questions  —  utility  and  constraints  —  mi’st 
await  the  collection  and  analysis  of  empirical  data. 

It  does  seam  appropriate,  however,  that  managers  would 
utilize  backlog  and  relationality  as  aids  in  day-to-day  managing  of 
resources  and  evaluation  of  operations  within  warning  centers  and 
groups.  With  respect  to  signification,  the  perspective  of  the 
authors  is  somewhat  different.  Intelligence  difficulties  caused  by 
problems  in  backlog  and  relationality,  which  in  turn  are  caused  by 
factors  such  as  resource  shortages,  constitute  one  type  of  difficulty, 
one  where  management  responsibilities  for  remediation  are  clear. 
However,  the  roots  and  causes  of  a  warning  failure  might  be  shown  to 
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derive  primarily  from  problems  in  epistemological  and  cognitive 
aspects  of  analysis,  reflected  in  a  system  of  analysis  whose  proce¬ 
dures,  models,  and  techniques  were  inadequate  to  the  situation.  Among 
other  things,  the  signification  measure  is  designed  to  illuminate 
these  kinds  of  problems.  Solutions  to  such  problems  will  in  part 
require  conceptual  and  technical  innovations  and  improvements  rather 
than  the  sort  of  adjustments  typical  of  every-day  management. 

Although  sharp  distinctions  among  the  sources  of  warning  problems 
are  not  always  likely,  it  would  be  useful  to  differentiate  the 
relative  importance  of  various  sources  of  difficulty.  Different 
problems  will  have  different  solutions,  some  of  which  will  be 
possible  given  simply  a  decision  (and  the  requisite  support)  to 
make  a  change;  others  will  require  further  conceptual  insight.  The 
real  purpose  of  the  signification  measure,  then,  is  to  provide  a 
a  basis  for  more  productive  inquiry  into  classes  of  subtle  problems 
in  warning  analysis.  (Presumably  it  has  been  clear  that  the  intent 
behind  the  signification  measure  is  not  the  second-guessing  of 
analysts.)  Knowing  about  the  problems,  we  must,  of  course,  proceed 
to  the  formulation  of  solutions. 

We  will  extend  these  comments  in  a  later  section  dealing 
with  analysis  management. 

Transition 

We  must  now  turn  to  the  implementation  of  the  system  of 
measures.  For  reasons  explained  below,  we  have  found  it  necessary 
to  design  a  system  of  analysis  procedures  and  techniques  whose 
enactment  by  analysts  will  create  t.ie  measures  data.  It  is  not 
entirely  a  new  system,  for  some  of  its  parts  derive  from  current 
operations.  But  one  point  is  very  significant:  our  system  follows 
our  model  of  warning  analysis,  builds  on  it,  depends  on  the  assump¬ 
tions  behind  it. 


V  - 
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There  is,  as  we  have  been  saying,  an  essential  connection 
between  the  way  analysis  is  done  and  how  it  is  measured.  The  key  is 
that  analysis  be  carried  through  from  monitoring  to  projection.  The 
measures  are  intended  to  determine  the  degree  to  which  this  is  done 
and  not  done  and  the  consequences. 

We  have  discussed  pathologies  in  information  systems  and 
the  importance  of  measures  to  diagnose  them.  We  can  close  for  now 
the  discussion  of  measures  and  proceed  to  analysis  as  a  method  by 
thinking  about  the  symptoms  of  incomplete  warning  analysis  in  the 
individual  and  the  system.  The  chief  symptom  is  memorably  suggested 
in  these  passages  by  the  writer,  Joan  Didion,  on  an  illness  she  has 
suffered.  Referring  to  certain  organic  disorders  of  the  central 
nervous  system,  Didion  writes: 


What  happens  appears  to  be  this :  ...  the  lining  of  a  nerve 

becomes  inflamed  and  hardens  into  scar  tissue,  thereby 
blocking  the  passage  of  neural  impulses  . . .  During  the  years 
when  I  found  it  necessary  to  revise  the  circuitry  of  my  mind 
I  discovered  that  I  was  no  longer  interested  in  whether  the 
woman  on  the  ledge  outside  the  window  on  the  sixteenth  floor 
jumped  or  aid  not  jump,  or  in  why.  I  was  interested  only 
in  the  picture  of  her  in  my  mind:  her  hair  incandescent  in 
the  floodlights,  her  bare  toes  curled  inward  on  the  stone 
ledge . 

In  this  light  all  narrative  was  sentimental.  In  this  light 
all  connections  were  equally  meaningful,  and  equally 
senseless.  I  was  meant  to  know  the  plot,  but  all  I  knew 
wa<-  what  I  saw:  flash  pictures  in  variable  sequence,  images 
with  no  'meaning'  beyond  their  temporary  arrangement,  not  a 
movie  but  a  cutting-room  experience. 
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We  must  now  turn  to  the  act  of  warning  analysis  itself. 

The  analytic  process  is  what  we  seek  to  measure,  manage,  and  improve. 

So  far  we  have  modeled  warning  analysis  aud  defined  measures  of  its 
performance.  At  this  point  we  must  design  means  of  implementing  the 
measures  in  warning  analysis  operations. 

Stated  very  simply,  this  is  our  basic  approach: 

1.  We  have  designed  the  structure  of  a  working  context 
for  warning  analysis.  By  context  we  mean  essentially  a  memory  of 
accumulated  analysis  against  which  new  data  is  analyzed  to  determine 
meaning.  The  structure  consists  of  twelve  forms.  The  forms,  highly 
changeable,  may  be  thought  of  as  a  language  for  various  kinds  of 
modeling.  The  forms  are  comprised  of  classes  of  flow  diagrams,  various 
types  of  mapping,  and  other  diagrammatic  structures.  As  such,  they  are 
especially  well-suited  for  computer-driven  display.  IWAMS  includes  a 
configurational  language  by  which  the  analyst  can  use  the  machine  to 
change,  recreate  or  develop  forms  rapidly.  Hence  there  is  flexibility 
within  the  internal  structure  of  the  forms:  idiosyncratic,  subjective 
aspects  of  analysis  are  accommodated.  Further,  new  meaning  and 
rationale  may  be  registered  easily.  Within  the  language  of  the  forms, 
then,  the  indi\idual  analyst  can  perform,  record  and  display  his  parti¬ 
cular  analysis. 

2.  The  twelve  forms  each  pertain  to  a  different  point  in 
the  model  of  the  warning  analysis  process.  For  example,  four  of  the 
forms  are  used  in  the  threat  recognition  stage;  several  others  are 
used  in  the  projection  stage.  Further,  the  forms  are  sequential: 
one  leads  logically  to  the  next,  and  so  on,  through  the  full  process 
of  warning  analysis.  They  are,  in  short,  an  interrelated,  progressive 
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series,  a  chain  of  outputs  of  different  analysis  steps  and  procedures 
Therefore,  the  language  of  the  forms  has  a  syntax. 

3.  Together  with  the  forms  we  have  designed  an  analytic 
routine ,  an  itinerary  of  interrelated  analysis  steps  whose  results 
are  localized,  focused,  recorded  and  displayed  in  the  context  of  the 
forms.  The  routine  is  designed  to  insure  that  the  process  of  analysi 
as  we  have  modeled  it  is  followed  appropriately.  Thus  there  is  an 
overall  grammar  of  analysis. 

4.  The  forms  and  the  routine  comprise  the  IWAMS  analysis 
system.  It  has  been  designed  to  enhance  analytic  effectiveness;  for 
example,  it  incorporates  the  use  of  anti -bias  techniques  and  other 
effectiveness-sponsoring  procedures  described  in  Section  3 . 

5.  The  entire  system  has  been  designed  to  be  highly 
applicable  for  computer-based  support.  The  extrasoma tic  memory 
based  on  the  forms,  and  the  analytic  routine,  are  designed  to  work 
against  the  psychophysiological  problems  discussed  above.  The 
approach  is  designed  to  foster  the  vital  heuristic  tendency.  In 
short,  the  system  gives  the  analyst  the  means  to  change  the  system. 

6.  The  performance  measures  are  implemented  simply  by 
using  the  system.;  When  forms  are  changed  in  certain  ways,  signifi¬ 
cation  is  evidenced .  Signification  is  recorded  and  displayed  by 
recourse  to  the  memory  of  forms.  The  memory  of  the  forms  becomes 
the  data  base  for  post  mortems .  By  following  the  routine,  and 
evidencing  this  by  computer  operations  such  as  calling  up  forms, 
relationality  is  pursued,  automatically  monitored  and  recorded  for 
derivation  of  measurement  data.  By  a  system  of  identifying  and 
tracking  the  flow  of  specific  input  data  through  the  routine  and  in 
relation  to  the  forms,  backlog  values  are  generated.  Hence  the 
objective  of  managing  analysis,  is  served  by  the  overall  process. 
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The  remainder  of  the  present  section  describes  the  IWAMS 
analysis  system  m  detail.  The  discussion  is  organized  around  the 
stages  of  analysis:  monitoring,  threat  recognition  and  projection. 
Each  is  discussed  separately.  For  each  stage,  the  following  topics 
are  stressed:  the  analysis  forms;  the  analytic  routine;  the  develop¬ 
ment  of  signification  values;  and  philosophical  aspects  of  machine 
support.  Following  that  a  description  is  given  of  an  informal  test 
at  a  U.S.  warning  center  of  a  preliminary  mock-up  version  of  IWAMS. 

In  part  the  test  is  described  in  order  to  progress  beyond  the  static 
nature  of  the  system  description  given  to  that  point  and  convey  a 
sense  of  how  IWAMS  would  function  operationally.  The  description  of 
the  test,  which  is  focused  on  the  North  Korean  warning  problem, 
includes  sequences  of  analysis  steps  and  procedures,  the  times  over 
which  they  occurred,  the  signification,  relationality  and  backlog 
values,  and  a  number  of  other  aspects.  Figure  7  shows  the  areas 
of  emphasis  in  the  discussion  of  methods  with  respect  to  the  overall 
IWAMS  concept. 

Before  proceeding  to  a  description  of  the  IWAMS  analysis 
system,  however,  it  is  essential  that  we  consider  two  fundamental 
subjects:  the  role  of  the  warning  analyst  and  the  contemporary 
crisis  of  interpretation.  Without  fully  appreciating  these,  it 
becomes  more  difficult  to  grasp  the  IWAMS  analysis  system. 


Profile  of  the  Warning  Analyst 

The  warning  analyst  watches  some  part  of  the  world.  He 
receives  information  of  many  kinds  and  levels  of  credibility  from 
many  sources  on  events  and  situations  in  his  sphere.  He  researches 
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problems  and  prospects  of  leaders,  countries,  alliances,  and  regions. 
Perhaps  he  has  visited  his  assigned  part  of  the  world.  He  may  have 
learned  one  or  more  of  the  languages  of  the  area.  He  has  undergone 
selected  training. 

The  warning  analyst  is  supposed  to  do  no  less  than  this: 
understand  the  dynamics  and  prospects  of  change  in  the  area  he 
watches;  recognise  signs  of  change,  particularly  threatening  change; 
assess  its  significance;  make  projections;  and  carry  out  warning  as 
appropriate. 

He  strives  for  literal  realism  but  properly  warns  on  the 
basis  oi  probabi.  ities . 

Through  modern  communications  technology,  he  encounters  on 
the  job  (and  off  the  job,  too)  an  unprecedented,  growing,  vast  amount 
of  information,  i  luch  of  it  individual  facts,  isolated  events.  He  is 
one  of  the  people  most  caught  up  in  the  information  revolution.  The 
myriad  data  provides  diverse,  sometimes  conflicting  views  of  parts  of 
reality. 


He  is  within  the  realistic  tradition  of  philosophy.  He 
is  an  empiricist.  His  operational  epistemological  assumption  is  that 
in  some  sense  "reality"  is  objective.  It  is  knowable.  There  are  at 
least  some  corre  spondences  between  his  experience  and  the  facts  of 
reality.  Moreover,  l:he  world  in  its  change  has  an  order,  governed 
by  sets  of  laws,  some  of  which  are  now  known  and  can  be  understood 
and  recognized  b/  humans  in  their  perception  of  reality. 

His  chief  impulse  is  interpretive.  The  analyst  will  be 
seeking  to  order  the  chaos  of  experience,  much  of  that  experience 
being  second-hand,  its  objects  removed  like  echos  from  the  scene  of 
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his  interest.  He  will  seek  to  discover  as  much  certainty  and  neces¬ 
sity  within  the  reality  he  watches  as  possible.  To  some  extent,  he 
must  believe  that  changes  in  the  area  of  the  world  he  watches  are 
approachable  in  terms  of  explainable  cause  and  effect.  If  change  is 
perverse,  it  is  also  consistent  within  some  bounds;  it  is  similar  to 
some  degree  across  different  times  and  places.  This  basic  logic  of 
change  derives  from  the  assumptions  of  purpose  and  of  reality  governed 
by  laws  recognizable  by  the  rational  mind  operating  on  current  and 
past  information.  Hence  factual  data  are  taken  as  reliable  cognitive 
elements . 


Perhaps  most  fundamentally,  he  will  believe  implicitly  in 
the  authenticity  of  models  organized  by  "plots";  models  in  which 
experience  is  ordered  in  chronological,  causal  forms.  This  is  not  to 
exclude  other  perceptual  and  analytic  modes  such  as  mosiac  techniques. 

Analyzing  input  data  with  his  models  and  analytic  techniques, 
perhaps  he  can  be  said  to  have  two  modes  of  interpretation: 

•  To  extract  a  pattern  of  significance  from  information 

•  To  impose  a  scheme  of  meaning  on  reality. 


The  Crisis  of  Interpretation 

In  the  view  of  some,  there  is  now  an  epistemological  crisis 
which  throws  into  doubt  our  interpretive  ability,  our  success  in  making 
sense  of  events,  our  ability  to  project  outcomes.  Two  of  the  sources 
of  the  crisis  most  frequently  cited  are,  first,  certain  discoveries 
in  twentieth  century  physics;  and,  second,  the  impact  on  hunans  of 
advances  in  communications  and  electronics  technology. 
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Within  the  present  scope,  we  can  do  no  more  than  briefly 
suggest  some  of  the  epistemological  impact  of  twentieth  century 
physics.  With  considerable  arbitrariness,  we  can  think  of  historical 
stages,  beginning  before  the  century  and  dating  back  to  Newton: 

Stage  1:  The  physics  of  Newton  produces  a  new  model  of 
reality,  a  new  epistemology.  A  scientific  view  of  the  real  world 
emerges  in  which  there  is  a  coherent  system  of  predictive  principles. 
In  the  eighteenth  century,  Laplace  claims  that  now  the  universe  can 
be  described  with  confidence;  that  it  is  predictable  with  certainty. 
Concepts  oi  man  in  his  perceptual  and  cognitive  relationship  to 
reality  tend  to  hold  that  there  are  "facts",  and  that  these  are 
epistemologically  reliable. 

Stage  2:  In  our  century  Einstein  throws  into  question 
Newton* c  gravitational  theory  and  its  perspective  on  temporal  and 
spatial  constraints,  its  reliance  on  laws  of  classical  mechanics. 

If  "facts"  were  previously  considered  reliable,  Einstein's  counter¬ 
intuitive  revolution  in  physics  now  creates  uncertainties.  In 
addition,  the  concept  of  the  elementary  particle  is  developed. 

Quantum  physics,  which  revises  the  principles  of  classical  mechanics, 
alters  the  older  notions  of  a  universe  ordered  in  a  predictable 
system  of  cause  and  effect.  Heisenberg's  concepts  of  uncertainty 
further  undermine  the  perspective  of  the  older  physics,  creating  a 
concept  of  a  microworld  of  particles  characterized  by  probability 
and  randomness.  Niels  Bohr’s  concepts  of  "complementarity"  justify 
an  accommodation  with  ambiguity.  Bohr  argues  that  opposing  theories 
of  the  nature  of  matter  are  valid  and  coexist  in  a  complimentary  way. 

Stage  3 :  The  scientific  realization  grows  that  there  is 
a  world  underneath  obvious  causality  which  is  not  graspable  by  current 
means  of  information;  that  earlier  deterministic  views  have  become 
suspect  at  the  very  least;  and  that  the  appropriate  attitude  seems  to 
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be  a  possibility  oriented  attitude  toward  reality.  In  the  final 
analysis,  the  new  theories  affect  man's  confidence  in  broad  terms. 
As  one  observer  has  written: 


The  behavior  of  the  particle  is  uncertain  and  therefore  the 
behavior  of  the  atom  is  an  uncertainty.  The  behavior  of  the 
atom  can  be  predicted  to  a  degree  of  probability.  The  behavior 
of  an  aggregate  of  atoms  is  therefore  only  a  probability,  and 
not  a  certainty.:  And  it  is  no  use  saying  that  the  degree  of 
uncertainty  is  too  small  to  effect  events  on  the  ordinary  scale, 
for  the  notion  of  determinism  is  similarly  based  on  the  funda¬ 
mental  determinism  of  the  individual  molecules,  multiplied  many 
times  to  become  the  world  of  nature. 


(George  Smiley)  hated  the  Press  as  he  hated  advertising 
and  television,  he  hated  mass  media  ,  the  relentless 
persuasion  of  the  twentieth  century.  Everything  he 
admired  or  loved  had  been  the  product  of  intense  individ¬ 
ualism. 

John  Le  Carre 
Call  For  The  Dead 

Perhaps  of  even  greater  significance  is  the  impact  of  the 
modern  communications  medic.  The  following  is  a  typical  view: 

The  new  communication  technologies  make  the  formulations  of 
any  encompassing  authoritative  visions  increasingly  more 
difficult,  since  they  produce  an  information  overload  which 
gives  such  diverse  and  disparate  views  of  reality  that  no 
single  interpretive  frame  can  contain  them  all  and  still 
present  a  coherent  vision  of  experience.  The  information 
revolution  also  expands  the  range  of  the  probable  to  the 
extent  that  it  blurs  the  boundaries  of  fact  and  fiction 

The  warning  analyst  obviously  is  victimized  by  the  informa¬ 
tion  deluge.  Modern  communications  media  have  created  a  sort  of  huge 
collage  added  to  daily  but  perversly  discontinuous.  It  is  a  collage 
of  facts,  images,  events,  and  "stories,"  large  and  small:  beyond 
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dcubt  it  is  an  information  overload  of  diverse  and  disparate  views 
of  reality. 

The  human  neurological  and  psychological  responses  to  this 
information  explosion,  particularly  in  professional  observers  and 
interpreters  such  as  journalists,  intelligence  analysts,  and  writers, 
is  sometimes  described  as  a  kind  of  pathology.  The  specific  problem 
becomes  the  inability  to  connect  things,  to  impose  order,  to  narrate 
process.  On  emotional  and  motivational  levels,  the  result  can  be  a 
variety  of  inappropriate  adjustments:  indifference;  escape  through 
reduction  of  the  interpretive  impulse  to  the  trivial;  demoralization; 
and  paranoia.  In  sum,  the  symptoms  of  a  crisis  of  interpretive 
confidence . 

Here  we  must  ask  the  reader's  indulgence,  for  the  impact  of 
media  on  the  interpretative  impulse  is  so  important  to  our  discussion 
that  we  must  briefly  explore  it.  Probably  the  most  dramatic  (if  among 
the  most  extreme)  symptoms  of  the  crisis  are  some  recent  developments 
in  American  literature,  developments  whose  significance  is  not 
unrelated  to  the  warning  analyst.  In  the  last  two  decades,  we 
encounter  some  novelists  who  blur  the  distinction  between  "fact"  and 
"fiction."  There  has  been  a  change  in  narrative  forms:  nonfiction 
novels,  transfiction,  surfiction,  and  other  new  forms  designed  to 
mix  the  real  and  the  fictional.  The  chief  reason,  as  one  literary 
observer  suggests,  is  the  view  that  "the  official  level  of  reality 
is  so  weird  that  what  passes  for  realistic  fiction  is  totally 
anachronistic,  like  an  old  road  map"  (essentially  a  variant  of  the 
old  adage,  "truth  is  stranger  than  fiction").  Thus  some  novelists 
view  their  role  differently  now:  they  want  to  be  "directly  involved" 
with  reality.  "Purely  imaginative"  works  can  no  longer  compete  with 
reality.  In  this  sense,  then,  the  writer  is  becoming  more  like  both 
the  journalist  and  the  intelligence  analyst. 
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A  dominant  source  of  these  new  perspectives  is  taken  to 
be  the  modern  communications  media.  The  media  have  amplified  the 
fictive  or  fantastic  quality  of  reality.  Because  the  major  media  are 
driven  by  economic  considerations  —  are,  in  fact,  in  showbusiness  -- 
the  news  tends  greatly  to  emphasize  "stories"  such  as  Vietnam,  the 
America  of  1968  (assassinations,  urban  ghetto  fires,  campus  wars, 
etc.),  Watergate  and  Iran  of  1978  and  1979.  There  is  judged  to  be 
a  circus  quality  about  the  major  media  reporting.  Through  the  mass 
media  v7e  are  used  to  seeing  images  —  usually  disconnected  and 
random  —  of  spectacular  events  and  people  caught  up  in  the  centers 
of  power,  which  in  turn  are  the  standard  stages  often  of  what  strike 
many  as  rather  fantastic  and  bizarre  situations  and  behavior. 

Because  of  time,  space  and  economic  constraints  on  reporting  and 
analysis,  the  output  of  media  remains  a  collage  whose  meaning  is 
difficult  to  determine.  Worse,  the  long-term  effect  is  to  condition 
one  to  the  present  and  to  the  superficial  explanation. 

Among  some  contemporary  writers,  the  reflective  tendency 
has  been  to  reduce  the  interpretation  of  reality.  Hence  even  thouqh 
they  are  becoming  more  "journalistic,"  these  writers  are  becoming  far 
less  interpretive  Writers  working  in  the  new  forms  tend  to  approach 
the  world  "as  it  is"  without  imposing  a  framework  of  meaning.  The 
French  writer,  Robbe-Grillet,  has  said  that  reality  as  experienced  by 
us  is  "neither  significant  nor  absurd.  If  is,  simply."  Some  modern 
literary  heroes  tend  t  end  up  either  having  lead  ambitious  expedi¬ 
tions  to  explore  reality  which  fail  to  yield  much  meaning,  or  simply 
having  decided  not  to  set  off  on  such  journeys  at  all.  Writers 
frequently  cited  as  working  in  the  new  forms  include  Thomas  Pynchon, 
nald  Barthelme,  John  Barth,  and  Steven  Katz. 

Such  writers  depart  the  tradition  of  classical  novelistic 
models  written  on  the  assumption  that  it  is  possible  to  have  an 
integrated  view  of  existing  realities.  Instead,  they  attempt  a 
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neutral  registration  of  experience,  rejecting  the  concept  of  art  as 
the  creation  of  order  from  chaos  and  the  writer  as  an  interpreter. 

Such  writers  do  not  approach  experience  in  search  of  cause  and  effect; 
they  do  not  organize  experience  linearly,  with  logical  development 
and  the  striving  for  resolution.  Techniques  of  collagd  are  more 
likely,  since  the  writers  are  opposed  to  more  traditional  novelistic 
forms  which  seek  to  perceive  and  interpret  a  whole  from  the  parts  of 
reality  experienced.  In  a  recent  study,  Mas'ud  Zavarzadeh  has  inter¬ 
preted  Thomas  Pynchon's  novel,  V.,  as  an  expression  of  the  mood,  the 
feeling,  of  such  writers  in  their  pessimism  about  meaning: 


Mistrust  and  even  fear  of  totalization  not  only  inform  the 
attitudes  of  innovative  American  narratists  today  but  also 
serve  as  their  immediate  subject  matter.  Such  dread  is  the 
basic  thematic  motif  of  Thomas  Pynchon’s  V.,  in  which  the 
protagonist,  Herbert  Stencil,  ostensibly  seeks  a  literal 
synthesis  of  the  numerous  confusing  manifestations  of  V. , 
but  actually  fears  any  solution  to  'the  V.  jigsaw..1 

(Stencil  wishes  to  avoid)  the  false  assurance  of  having 
obtained  the  harmonizing  principle  behind  the  manifold 
reality.  Stencil's  fear  of  piecing  together  the  hints, 
clues,  and  signs  he  has  come  across  during  his  search  - 
caused  by  his  suspicion  that  these  are  all  planted  in  his 
way  by  some  conspiracy  to  mislead  him  and  prove  that  his 
integration  of  V.  is  no  more  than  merely  a  scholarly  quest 
for  a  synthetic  wholeness  -  is  opposed  by  the  equal  weight 
of  his  anguish  in  not  being  able  to  locate  V.  and  separate 
her  from  various  V.-like  appearances.  His  conflicting 
feelings  are  indications  that,  while  he  has  a  strong 

yearning  for  a  unifying  belief,  the  norm  in  his  life  has 
so  unmistakably  become  the  fragmentation  and  chaos  of 
experience  that  even  the  thought  of  reaching  a  cohesive 
vision  of  the  wholeness  beneath  the  scattered  surfaces  is 
not  only  suspect  but  also  phony  This  agonizing  impasse  is 
characteristic  of  contemporary  man's  compulsive  attempt  to 
identify  a  solid  core  of  trustable  reality  distinguishable 
from  fictitious  appearances  and  his  mistrust  that  the  pattern 
he  comes  up  with  may  be  a  mere  projection  based  on  no  more 
than  recurrence  of  an  initial  and  a  few  dead  objects  thrown 
in  his  way  by  cabals .:  As  one  character  in  V,  states:  'In  a 
world  such  as  you  inhabit,  Mr.  Stencil,  any  cluster  of 
phenomena  may  be  a  conspiracy . ' 
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Ultimately,  reality  is  viewed  as 


...  a  form  of  organized  chaos,  resisting  interpretation 
and  inherently  indeterminate  to  such  an  extent  that  a 
paralogic  of  conspiracy  is  the  only  kind  of  reasoning  which 
can  account  for  its  strangeness. 

Most  importantly,  a  profound  tension  arises  because  one  of  our 
strongest  impulses  is  the  interpretive  impulse. 


Whether  we  can  say  that  today's  history  is  really  any  more 
bizarre  than  earlier  times  with  their  plagues,  religious  wars, 
ironic  expeditions  and  rapacious  colonizations,  is  perhaps  uncertain 
But  surely  twentieth  century  technology  has  made  the  world  smaller, 
more  dangerous.  And  clearly  the  "age  of  information"  and  "the  data 
market"  have  resulted  in  an  information  overload  that  is  peculiarly 
perverse.  It  seems  beyond  question  that  the  interpretive  impulse 
has  been  blunted.  Confidence  in  the  power  of  the  imagination  to 
simulate  reality  and  lead  to  und  ’•standing  has  diminished. 

It  suiis  our  purpose  here  to  take  account  of  the  interpre¬ 
tive  crisis,  to  recognize  it  as  a  major  obstacle  to  be  overcome. 
Certainly  the  warning  analyst  does  not  live  in  a  vacuum;  he  also 
is  affected  by  the  general  environment.  Thus  while  we  have  been 
reviewing  some  developments  in  literature  to  suggest  quickly  the 
general  crisis  of  intepretation,  we  should  note  that  symptoms  may 
sometimes  appear  in  the  world  of  the  warning  analyst  as  well,  such 
as : 


•  Skepticism  (really,  pessimism)  at  the  prospects  for 
more  sophisticated  analytic  methods  and  approaches . 
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•  Extensive  emphasis  on  indicators  and  monitoring  to  the 
neglect  of  modeling  and  projection.  This  is  sometimes 
referred  to  as  lack  of  proper  emphasis  on  the  esti¬ 
mative,  longer-term  aspect  of  warning  analysis. 

•  Narrow  de  facto  perspectives  on  warning  as  essentially 
a  very  short  term  activity  with  virtually  exclusive 
emphasis  on  military  dimensions. 

•  Less  than  adequate  funding  for  warning  operations 
and  R&D. 

The  Psychophysiological  Basis.  There  is  another  view  of 
the  interpretive  crisis,  one  that  implies  a  technological  response. 
Let  us  consider  that  the  information  deluge,  which  is  here  and  must 
be  dealt  with,  is  an  opportunity.  The  warning  analyst  stands  to 
benefit  if  through  conceptual  and  technical  means  we  become  able  to 
exploit  the  unprecedented  information  available  today  for  analysis. 

It  is  most  important,  however,  to  consider  some  of  the  reasons  why 
we  have  not  done  so.  We  can  attempt  an  explanation  as  follows: 

1.  At  least  in  a  time-critical  sense,  the  unaided  human 
mind  seems  unable  to  process,  and  then  conduct  extensive  interpre¬ 
tive  cognition  on,  the  massive  modern  information  input.  With 
reference  to  our  model  of  warning  analysis,  the  analytic  process 
is  truncated;  there  is  a  tendency  not  to  achieve  follow-through, 
not  to  complete  the  full  interpretive  thrust. 

2.  If  the  information  input  is  accurately  described  as  an 
overload,  perhaps  this  is  largely  in  relation  to  the  unaided  mind.. 


3.  For  reasons  given  above,  in  the  cognitive  process  of 
analysis  we  forget  as  we  learn.  Therefore  we  have  great  difficulty 
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in  establishing  continuity  and  accuracy  in  the  memory,  the  recon¬ 
struction  through  post  mortems ,  of  meanings  and  rationales.  This 
condition  is  continuously  exacerbated  by  the  intensity  and  peculiar 
features  of  the  modern  information  input. 

4 .  The  mind  must  have  recourse  to  models  —  schemata  — 
through  which  it  filters  the  input  data  in  order  to  make  sense  of 
it.  But  the  problem  described  in  item  1  has  resulted  in  a  failure 
to  build  sophisticated  analytic  contexts  for  analysis  and  interpre¬ 
tation  of  new  data. 

5.  The  schemata  arising  in  the  mind  are  both  inborn  and 
learned.  Learned  schemata  are  the  major  analytic  tool,  a  major 
response  of  the  brain,  in  operating  on  indirect  and  second-hand  data, 
which  are  the  classic  inputs  to  the  warning  analyst  who  typically 
receives  reports,  messages,  and  visual  data  on  remote  places,  persons, 
and  events. 

6.  Three  additional  problems  now  ensue.  The  first  is  a 
problem  in  time.  In  the  face  of  persistent  input  overload,  there 
may  be  insufficient  time  —  literally  no  pause  —  for  the  unaided 
individual  human  analyst  to  develop  and,  more  importantly,  refine 
appropriate  schemata.  Schemata  are  required  that  are  refined 
enough  at  least  to  give  promise  of  dealing  with  some  effectiveness 
with  the  awesome  challenge  of  warning  analysis:  the  interpretive 
task  of  attempting  to  understand  remote  parts  of  the  world  with 
literal  realism,  recognizing  threats  (including  novel  threats), 
and  at  least  warning  appropriately  on  probability.  This  is  not  to 
broach  the  more  fundamental  problem  of  the  constraints  on  the 
unaided  brain  of  reconstructing  variations  in  past  schemata. 

7.  The  second  problem  is  this:  even  if  there  were  time, 
the  unaided  human  mind,  no  matter  how  heroic  and  endowed  with  genius, 
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probably  would  not  invent  numerous  great  schemata,  for  all  the 
reasons  given  previously,  all  the  constraints  on  schemata  preserva¬ 
tion  and  storage,  and  hence  refinement,  which  arise  from  the  nature 
of  the  mind's  operation  on  data. 

8.  The  result  of  these  problems  is  a  specific  shortfall: 
the  unaided  mind  has  difficulty  achieving  a  certain  kind  of  stasis. 

As  we  have  said,  hindsight  analysis  and  post  mortems  are  key  to 
analysis  and  its  measurement  and  management.  Management,  of  analysis 
begins  with  the  individual  analyst  managing  his  own  analysis;  post 
mortems  and  hindsight  analysis  should  not  be  the  purview  only  of 
committees  exploring  intelligence  failures  but  should  be  part  of 
analysts'  day-to-day  attack  on  the  warning  problem.  For  this  there 
must  be  stasis,  a  freezing  of  past  situations  of  signification, 
those  occasions  when  the  analyst  (or  analysts)  assigned  meaning  by 
operating  cognitively  on  the  data  through  specific  rationales.  But 
as  we  have  seen,  signification  capturing  will  not  be  adequately 
performed  by  the  unaided  mind. 

9.  The  third  problem,  which  we  will  cite  here  and  discuss 
below  in  a  more  suitable  context,  is  the  difficulty  of  the  mind  in 
rapidly  perceiving  the  far-reaching  ramifications  (or  implications) 
of  judgments  which  initially  change  single  elements  or  a  few  elements 
in  complex  warning  problems  whose  modeling  involves  many  elements 
interrelated  in  a  coherent  framework.  Initial  changes  to  single 
elements  may  many  subsequently  cause  shifts  in  likelihoods  at  various 
locations  in  the  model  structure.  In  the  case  of  a  complex  set  of 
interrelated  schemata,  the  shifts  could  create  structural  changes  at 
many  points. 


This  brings  us  to  what  the  authors  believe  are  the 
beginnings  of  a  resolution  for  tnese  problems:  the  IWAMS  analysis 
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system  and  its  associated  analytic  performance  measures .  Both  were 
summarized  above.  We  will  now  undertake  a  detailed  discussion.  In 
doing  so  it  is  important  to  note  several  points . 

The  IWAMS  analysis  forms  and  routine  take  into  account 
existing  operations  at  warning  centers  as  observed  by  the  authors. 
For  the  threat  recognition  and  projection  stages,  however,  the 
authors  have  themselves  fashioned  forms  and  a  routine.  A  number  of 
methods  of  analysis  developed  in  fields  outside  intelligence,  but 
believed  highly  adaptable  and  useful  for  a  first  generation  method¬ 
ology,  have  been  used.  It  is  strongly  emphasized  that  the  analytic 
forms  and  routine  are  considered  a  first  generation  approach  that 
would  be  changed  and  refined  by  analysts  during  operational  use. 

One  of  the  major  premises  in  IWAMS  is  the  progressive  improvement 
of  analysis  through  the  warning  heuristic.  The  main  reason  for  the 
analytic  routine  is  the  presumption,  active  at  every  point  in  the 
present  research,  that  rigorous  analysis  reguires  a  system  of 
warning  analysis  based  on  the  model  of  warning  analysis  Against 
this  background,  we  may  now  discuss  the  IWAMS  analysis  system  in 
terms  of  the  individual  stages  of  warning  analysis. 


Monitoring  Stage 

In  conducting  the  day-to-day  watch  on  an  area,  the  warning 
analyst  monitors  the  activit"  ..evels  of  numerous  indicators. 

Typically  there  are  preestal  sued  lists  of  indicators  of  military, 
political,  economic,  and  cul.  .ral  events.  These  are  judged  to  be 
necessary  and  significant  events  in  processes  leading  to  the  enactment 
of  threats.  For  example,  indicators  may  be  developed  and  organized 
to  show  increases  in  the  readiness  of  military  forces  to  launch  an 
attack.  Political  indicators  may  be  developed  and  monitored  to 
provide  early  warning  of  potential  crises  through  detecting  the 
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emergence  and  character  of  political  hostility  between  or  among 
nations  or  factions. 

Indicators  are  generally  categorized.  For  example,  military 
indicators  may  be  subdivided  into  such  categories  as:  unusual  deploy¬ 
ments  of  military  forces;  intensive  logistics  activity;  military 
exercises  conducted  at  atypical  times;  and  similar  examples. 

There  has  been  considerable  indicator  research  in  warning 
and  crisis  management  over  the  pur''  decade.  The  research  has  led 
to  definition  and  development  of  indicators  and  to  design  of 
computer-based  analysis-aid  systems.  The  analyst  might  use  computer- 
based  aids  which  operate  on  large  volumes  of  indicator  data  and 
produce  quantitative  output  on  activity  levels,  for  example, 
the  number  of  adversary  ships  in  or  out  of  position  by  specific 
geographic  area.  Given  the  input  of  indicator  data  from  a  variety 
of  sources,  the  basic  purpose  of  such  aids  would  be  to  determine 
to  what  extent,  based  on  the  norms  inherent  in  accumulated  data, 
there  are  unusual  activities  present;  and  the  likelihood  of  various 
threatening  trends  developing.  Judith  Ayres  Daly  of  the  Defense 
Advanced  Research  Projects  Agency  describes  one  monitoring  aid  now 
under  development  which  involves  political  indicators: 

The  Early  Warning  and  Monitoring  System  is  a  computer-based 
interactive  global  I&W  system.  It  is  based  on  quantitative 
political  indicators  which,  along  with  probabilistic  forecasts, 
are  displayed  in  graphical  or  tabular  form.  (The  system)  is 
being  improved  by  the  addition  of  rigorous  forecasting 
methodologies  and  automation  of  many  of  the  necessary  but 
time-comsuming  tasks  which  must  be  performed  by  an  ISM 
analyst.  These  include  automatic  generation  of  hotspot, 
alert,  and  monitoring  lists  which  are  designed  to  supplement 
the  analyst's  other  sources  of  information  and  save  him  time 
by  focusing  his  attention. 
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Clearly  such  approaches  and  technology  tend  to  mitigate 
problems  of  information  overload  and  to  enable  analysts  to  realize 
the  opportunities  inherent  in  the  availability  of  large  volumes  of 
data. 


Indicators  are  never  very  far  from  threat  models.  This  is 
no  surprise  since  the  analyst  eventually  will  seek  to  make  sense  of 
the  activity  of  indicators  within  some  context.  If  indicators  are 
listed  in  a  sequence  believed  to  represent  a  probable  chronology  of 
necessary  and  significant  adversary  preparations  for  attack,  the 
indicators  in  effect  have  been  assembled  into  a  model.  In  the 
present  research,  threat  modeling,  which  occurs  in  the  threat 
recognition  stage,  should  incorporate  indicators  associated  with 
the  monitoring  stage.  At  this  point,  we  should  proceed  to  a 
consideration  of  the  threat  recognition  stage  and  discuss  it 
together  with  monitoring.  The  close  relationship  between  the  two 
will  become  clear. 


Threat  Recognition  Stage 

As  the  anthropologist,  Edward  T.  Hall,  has  said,  "Man  is 
the  model-making  organism  par  excellence . 11  In  the  threat  recognition 
stage,  the  analyst  now  faces  a  much  greater  challenge  to  his  imagin¬ 
ation  and  general  analytic  ability,  for  he  is  attempting  to  model 
potential  courses  of  action  by  various  countries,  entities,  and 
decisionmakers  in  his  area  of  interest.  These  models  act  as 
filters  through  which  he  reviews  input  data,  particularly  data  on 
the  activity  of  indicators  from  the  monitoring  stage. 

The  most  frequently  used  term  for  such  models  is  scenario. 
The  warning  analyst  is  indeed  a  scenarist.  He  must  imagine  hypo¬ 
thetical  situations  as  an  essential  part  of  the  process  of  analysis. 
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He  must  use  his  experience  captured  in  his  memory,  and  the  exper¬ 
ience  of  others  in  a  variety  of  forms,  in  creating  the  situation. 

The  analyst  must  be  a  model  builder  because  the  situation 
(not  the  event)  is  the  most  meaningful  unit  of  warning  analysis.  The 
major  goal  of  warning  analysis  is  to  recognize  a  potential  threat 
situation  prior  to  the  full  onset  of  the  impact.  The  term, 
situation,  is  of  special  importance.  Situations  are  considered  as 
"things"  to  be  recognized  by  analysts.  Situations  may  be  defined 
as  consisting  of  three  stages:  (1)  the  present;  (2)  processes 
leading  to  an  outcome  or  situation  result;  and  (3)  the  result 
itself  (which  can  be  an  adverse  impact  on  a  given  U.S.  asset). 

The  analytic  process  should  result  in  recognition  of  a  specific 
threat  situation  in  advance  of  the  full  outcome.  The  process  can 
be  shown  as  follows: 


RECOGNITION  PERIOO 


I 


It  follows  from  the  primacy  of  the  situation,  itself  a  complex 
arrangement  of  components,  that  the  analyst  becomes  a  model  builder, 
a  scenarist:  it  is  how  he  must  envision  and  analyze  situations. 

Here  the  concept  of  forms  becomes  important. 

Analysis  Forms.  We  now  turn  to  specific  examples  of  analytic 
forms.  As  a  hypothetical  case,  we  will  focus  on  the  problem  of  warning 
of  hostile  activity  directed  against  U.S.  interests  and  assets  by 
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North  Korea.  A  set  of  four  interrelated  analysis  forms  believed 
useful  in  monitoring  and  threat  recognition  have  been  developed  as 
follows : 


•  Projected  Alternative  Major  National  Courses  of 
Action  ( PAMNACS ) .  This  form  is  a  structure  for  expressing  what 
are  believed  the  most  definitive  national  policies/courses  of 
action  of  a  country  such  as  North  Korea  with  respect  to  key 
warning  aspects,  such  as  "reunification"  of  the  Korean  peninsula 
and  the  international  position  of  North  Korea  vis  a  vis  ROK, 

Japan,  U.S.,  Europe,  PRC,  U.S.S.R.,  and  other  countries.  Figure  8 
shows  an  example  of  the  FAMNACS  form  used  for  projection  of  some 
alternative  major  courses  of  action  by  North  Korea.  Several 
possible  courses,  ranging  from  negative  to  positive  impact,  are 
identified.  Certain  courses  can  be  considered  as  leading  to 
possible  crisis  situations.  Note  that  the  PAMNACS  form  is  compat¬ 
ible  with  a  system  of  computer-driven  displays. 

The  PAMNACS  should  be  as  comprehensive  as  the  analyst  (or 
group  of  analysts)  believes  necessary.  Thus  PAN  MACC  are  always 
and  readily  changeable .  Figure  9  is  a  further  example  in  which  two 
alternative  courses  of  action  by  North  Korea  are  expanded  beyond  the 
first  version  of  PAN  MACS.  The  two  alternatives  are,  first,  that 
North  Korea  attempts  to  unify  the  peninsula  by  war  prior  to  any  U.S. 
military  withdrawal  (a  withdrawal,  say,  of  the  scope  contemplated 
early  in  the  Carter  adminstration) ;  and,  second,  that  the  North 
Koreans  seek  to  unify  the  peninsula  by  war  after  such  a  withdrawal. 
Let  us  imagine  that  a  study  has  been  made  of  the  various  forms 
such  eventualities  might  take.  These  forms  could  include:  a 
protracted  conflict;  a  quick  strike  by  the  North  against  key  targets 
in  the  South  and  the  seizure  of  these  targets,  leading  presumably 
to  eventual  demoralization  of  the  ROK  and  its  capitulation;  and  a 
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limited  strike  by  the  North,  for  example,  against  off-shore  islands, 
designed  to  foment  tension  and  unrest  in  the  ROK  which  leads  to  a 
new  government  in  the  South  willing  to  consolidate  with  P'yongyang. 

The  specific  examples  giver-  here  are  not  as  important  as  the  principle 
that  PAMNACS  must  be  flexible;  that  it  must  accommodate  new  analytic 
perspectives  as  they  arise. 

•  Decision/Event  Networks  (DEN).  Figure  10  shows  an 
example  Decision/Event  Network,  the  second  type  of  form.  DENs  are 
extensions  of  the  PAMNACS.  For  each  of  the  possible  major  national 
courses  of  action  shown  in  the  PAMNACS,  a  DEN  is  developed  and 
serves  as  a  means  of  further  modeling  the  possible  activity  that 
North  Korea  might  exhibit  in  implementing  that  particular  course 
of  action.  Figure  10  refers  to  a  specific  course  of  action  by  the 
North  in  which  it  seeks  reunification  with  the  South  by  military 
means  after  a  U.S.  withdrawal;  and  plans  to  do  so  by  means  of  a 
protracted,  all-out  war.  Figure  11  shows  a  DEN  for  the  same  course 
of  action  but  under  a  different  military  plan:  the  quick  strike 
option.;  Similar  DENs  would  be  developed  for  remaining  elements  in 
the  PAMNACS.  Of  course,  DENs  are  always  changeable. 

The  events  and  activities  in  the  DEN  are  developed  on 
the  basis  of  the  indicators  from  the  monitoring  stage  of  analysis. 

One  of  the  major  purposes  of  the  forms  in  the  threat  recognition 
stage  is  to  provide  a  context  for  the  indicators. 

Tne  numbers  shown  at  the  nodes  of  the  DENs  are  for  purposes 
of  indexing  and  referencing  the  next  type  of  form,  the  critical  event 
filter. 


•  critical  Event  Filters  (CEF).  Figure  12  shows  an 
example  of  a  CEF  that  applies  to  r.he  PAMNACS.  Similar  examples 
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Figure  12 .  Example  CEF 
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would  apply  to  the  DENs.  CEFs  may  be  thought  of  as  containers 
for  holding  data  that  are  judged  to  signify  that  the  activity 
modeled  at  given  nodes  in  the  PAMNACS  and  in  the  DENs  is  occurring. 
For  example,  with  reference  to  Figure  11,  Node  23,  input  data  on 
the  convergence  of  North  Korean  ground  units  in  areas  outside 
their  garrisons  could  signify  the  assembly  of  combat  troops  and 
hence  render  that  node  active.  In  a  computer-based  memory,  the 
CEFs  could  be  considered  a  portion  of  the  memory  in  which  such 
input  data  would  be  stored. 


•  Anomalous  Event  Matrix  (ANEM).  Although  always  a 
goal,  obviously  it  will  not  be  possible  to  model  all  threat 
contingencies  in  advance.  We  have  discussed  the  reasons  for  this. 
We  know  that  novel  situations  will  develop  and  that  recognizing 
them  is  one  of  the  major  challenges  to  warning  analysts. 


It  is  important  to  be  alert  to  those  cases  in  which 
incoming  data  does  not  match  well  with  the  preestablished  context. 

In  these  cases,  the  analyst  should  have  learned  that  he  must  take 
the  precaution  of  formulating  new  (or  novel)  threat  parameters  in 
order  to  detect  any  situational  patterns  developing  in  the  data. 
Procedures  for  developing  new  threat  parameters  will  generally 
follow  those  for  designing  threat  models  per  se.  The  analyst  should 
be  particularly  sensitive  to  his  limitations  in  knowledge  and 
expertise,  and  insure  that  appropriate  technical  consultation  is 
obtained  if  there  appear  to  be  "abnormal1’  or  inexplicable  relation¬ 
ships  in  the  data. 


The  analytic  form  denoted,  Anomalous  Event  Matrix,  is  one 
approach  to  aiding  in  the  difficult  task  of  recognizing  novel  threat 
situations  and  of  dealing  with  new  data.  The  ’echnique  involves 
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special  examination  of  input  data  which  does  not  relate  satisfac¬ 
torily  to  established  filters.  The  examination  consists  of 
developing  a  simple  matrix  in  which  input  data  is  compared  against 
new  hypotheses;  possible  changes  in  technical  capabilities; 
possible  new  policies  in  a  foreign  country;  etc.  The  intended 
result  is  to  develop  new  perspectives  which  tie  tne  input  data 
together  better  then  the  existing  ones.  In  turn,  these  new 
perspectives  can  be  used  to  modify  or  expand  the  existing  models 
or  to  create  new  ones.  This  analytic  process  is  of  enormous 
significance  in  several  dimensions,  as  discussed  below. 

Analysis  Routine.  The  forms  and  associated  modeling 
involve  an  analysis  routine.  The  routine  is  ultimately  based  on 
the  functional  model  of  analysis  in  the  threat  recognition  stage 
as  described  in  Section  2.  That  stage  consists  of  four  major 
analytic  steps  and  17  distinct  analytic  procedures  with  the  input 
data.  We  must  now  relate  the  modeling  to  the  routine. 

We  should  begin  in  general  terms  by  considering  the  basic 
thrust  of  the  analysis  itself.  That  thrust  is  to  search  for,  and 
discover,  meaning  in  input  data  by  relating  it  to  a  context  of  models 
through  the  use  of  certain  procedures.  Here  the  analyst  is  embarked 
on  the  analytic  adventure,  a  difficult  mental  expedition  taxing  his 
full  powers  and  demanding  that  he  try  to  overcome  human  tendencies 
that  work  against  his  analysis .  In  theory  the  analyst  should  not 
be  satisfied  until  he  can  account  for  data,  can  assign  it  meaning. 
Perhaps  the  pervasive  benefits  of  full  analysis  pursued  systemati¬ 
cally  and  with  rigor  —  in  short,  using  a  routine  --  can  best  be 
suggested  by  looking  at  the  problem  of  novel  threats. 
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We  said  above  that  the  analyst  must  be  alert  to  those 
cases  in  which  input  data  does  not  match  well  with  the  system  of 
preestablished  models.  Conceivably  these  cases  may  signify  a  novel 
threat  situation.  At  the  very  least  they  may  signify  that  the  present 
versions  of  PAMNACS,  DENs,  and  CEFs  are  too  narrow  to  account  for  the 
perceived  events  and  activities.  Therefore,  the  effective  assignment 
of  meaning  cannot  be  accomplished  until  the  models ,  and  perhaps 
procedures  as  well,  are  expanded  or  modified.  Put  another  way:  the 
analyst  cannot  effectively  analyze  such  data  all  the  way  through  the 
full  process  until  he  invents  new  structure  and  analytic  strategies 
to  do  so.  In  the  regenerative  process  of  doing  this  —  of  responding 
to  a  problem  of  signification  —  the  analyst  may  discover  a  new  threat, 
a  new  dimension  to  the  problem,  a  new  indication,  and  so  on.  This 
will  create  analyst-inspired  change  to  the  structure  of  the  analytic 
system  itself.  Through  the  changes  he  makes,  the  analyst  will  tend  to 
mitigate  problems  of  entropy  and  to  refine  methods  of  analysis.  He 
will  tend  to  stay  more  current  on  the  problem  area  he  is  responsible 
for.  In  sum,  if  the  interpretive  impulse  is  facilitated,  not  blunted, 
the  analyst  has  the  chance  to  be,  and  to  become  increasingly,  effec¬ 
tive.  The  paradox,  profound  and  subtle,  is  this:  only  by  working 
systematically  and  hard  to  create  a  set  of  imperfect  models  does  the 
analyst  create  the  basis  for  improving  his  effectiveness.  His  chance 
to  improve  his  effectiveness  lies  in  his  interpretive  impulse  which 
causes  him  to  worry  the  new  data  against  the  imperfect  models,  out  of 
a  mixture  of  skepticism  and  excitement,  and  to  make  refinements.  The 
analytic  routine  helps  him  to  be  thorough.  The  context  of  models 
helps  him  to  see  when  he  knows  something  and  may  be  right;  and  when 
he  does  not  and  must  create  ways  to  do  so.  As  we  have  said  earlier, 
he  must,  because  of  the  ways  in  which  his  mental  capabilities  have 
evolved,  rely  on  extrasomatic  memories  and  extrasoma tic  analytic 
support  in  accomplishing  these  goals. 
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We  should  consider  briefly  certain  other  aspects  of  the 
modeling  process  in  warning  analysis,  for  when  the  analyst  follows 
the  threat  recognition  routine  he  does  much  more  than  is  expressed  in 
the  individual  steps  and  procedures  outlined  in  the  model.  Indeed, 
among  the  unique  aspects  of  warning  analysis  are  the  special  roles 
of  memory  and  imagination.  In  warning  analysis  there  is  a  crucial 
requirement  for  effective  memory.  An  analyst  must  recall  associa¬ 
tions  and  relationships,  past  behaviors,  and  past  events  in  order  to 
understand  current  information  as  a  basis  for  projecting  the  future. 

We  remember  by  recourse  to  our  information  base  of  actually  sensed 
events.  Further,  memory  is  colored  by  imagination:  given  that  the 
mind  simulates  reality  from  the  abstractions  of  sense  impressions 
it  can  equally  well  simulate  reality  by  recall  and  fantasy,  creating 
stories  and  playing  imagined  and  remembered  events  back  and  forth 
through  time.  Obviously  these  powers  apply  to  the  modeling  of  hypo¬ 
thetical  threat  situations  (as  well  as  to  projections)  in  the  process 
of  warning  analysis.  It  may  seem  a  grimly  humorous  question  in  light 
of  certain  history,  but  is  the  analyst  doing  anything  fundamentally 
different  when  he  makes  such  projections  as  against  when  he  daydreams? 
In  purpose  and  mental  direction,  the  answer  is  (or  should  be):  yes, 
he  is  doing  something  very  much  different.  But  in  terms  of  essential 
mental  processes,  he  is  not. 

Hence  we  must  distinguish  various  kinds  of  imaginative 
activity.  With  some  wariness,  we  might  consider  disciplined  versus 
undisciplined  imagining.  If  one  is  fantasizing  a  pleasant  and 
personal  future  situation  —  and  all  fantasizing  ultimately  implies 
the  future  —  then  one  is  likely  co  be  in  a  kind  of  meandering,  free- 
associarion  mode.  In  some  psychology  this  process  is  exploited  for 
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therapeutic  purposes.  In  some  literature  it  is  modeled  in  what  is 
called  the  ’’stream  of  consciousness"  tradition  in  an  attempt  to 
convey  the  sense  of  a  human  being  by  showing  the  patterns  and  tenden¬ 
cies  in  this  ruminating  use  of  the  imagination.  If  there  are  problems 
of  mindset,  bias,  wishful  thinking,  distortion,  mirror  imaging,  and 
all  the  rest  of  the  catalog  of  pitfalls  in  human  perception  and  cogni¬ 
tion,  it  is  good  that  the  problems  arise  and  are  evident,  both  for 
the  therapist  and  for  the  writer  who  is  trying  to  dramatize  ^  parti¬ 
cular  human  personality. 

But  man  the  analyst  must  do  directed  threat  modeling  and 
projections,  which  of  necessity  must  be  done  through  "disciplined" 
imagination.  They  must  be  done  imaginatively  because,  as  we  have 
said,  the  human  being  has  no  other  way  of  challenging  the  future 
except  by  going  back  through  the  data  base  of  remembered  images, 
scenes,  hypotheses,  convictions,  etc.  and  creatively  putting  toge ther 
that  future.  The  sobering  aspect  is  that  warning  analysis  has  as 
its  basic  thrust  the  Sisyphean  attempt  to  understand  the  future  with 
literal  realism,  with  great  specificity  across  all  the  information 
categories:  who,  what,  when,  where,  how,  and  why.  It  seeks  und^1" 
tremendous  constraints  to  model  and  project  what  will  happen  both 
in  an  unimpeded  sense  and  an  influenced  sense.  Warning  analysis 
thus  becomes  a  uniquely  demanding,  creative  act.  Hence  the  routine 
becomes  essential  for  disciplined  approaches. 

When  a  novelist  or  playwright  or  movie  director  creates  an 
idealized  wc  -Id  full  of  people,  events,  conflict,  and,  ultimately, 
meaning  (even  if  it  is  the  view  that  there  is  no  meaning),  a  projec¬ 
tion  of  sorts  has  been  made.  An  enormously  complex,  sophisticated 
body  of  technique  and  narratology  —  all  of  it  highly  disciplined  — 
has  grown  up  around  the  making  of  novels  and  plays  and  movies.  On  a 
different  level,  the  warning  analyst  is  also  a  narratist.  There  are 
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general  techniques  of  narratology  highly  germane  to  warning  analysis 
in  modeling  and  projection. 

The  warning  analyst  undertakes  to  develop  scenarios.  What 
are  the  techniques  and  stylistic  features  of  the  scenario?  The 
ability  to  construct  realistic  and  plausible  scenarios  presupposes 
not  only  a  comprehensive  knowledge  of  an  area  and  its  possible  trends, 
but  also  demands  varying  degrees  of  relatively  specialized  literary 
skills.  The  basic  problem  is  to  take  the  essential  ingredients  of  a 
hypothetical  situation  and  order  them  into  a  coherent,  realistic 
sequence  of  events.  Existing  scenarios  will  require  periodic  refine¬ 
ments  to  lend  them  greater  impact  and  credibility  on  the  basis  of 
new  data.  This  process  is  also,  of  course,  part  of  the  dynamic  of 
signification. 

The  analytic  forms  described  above  are  designed  for 
flexibility  in  scenario  development.  One  of  the  basic  advantages 
of  scenarios  lies  in  their  diversity.  They  can  vary  considerably, 
both  in  subject  matter  and  form.  However,  the  essential  nature  of 
any  scenario  is  that  the  situation  described  is  hypothetical.  By 
definition,  a  scenario  is  a  depiction  of  a  hypothetical  series  of 
events  that  could  produce  some  envisaged  situation.  Thus  the 
analyst's  basic  problem  is  to  establish  a  coherent  pattern  of  events 
within  which  the  projected  situations  could  plausibly  occur.  In 
doing  so,  he  is  establishing  structures  of  rationale.  These  in  turn 
are  bound  up  in  signification. 

Much  has  been  written  about  achieving  plausibility  in 
writing  and  in  other  art  forms  that  are  largely  imaginative  (or 
hypothetical).  In  its  broadest  dimensions,  the  problem  is  that  of 
using  all  the  elements  of  composition  effectively:  mechanical 
elements  such  as  narration  and  description?  and  conceptual  elements 
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such  as  plot  and  unity.  This  challenge  entails  disciplined  approaches 
However,  the  great  majority  of  scenarios  fall  within  relatively  narrow 
limits  as  to  the  mechanical  and  conceptual  problems  they  pose  and 
thereby  require  rather  straightforward  routines.  One  reason  is  that 
scenarios,  though  they  need  not  do  so,  are  generally  intended  to  serve 
as  broad  outlines  of  likely  processes.  Their  characteristics  include 
the  following: 


•  Scenarios,  unlike  dramatic  fiction  and  informal  essays, 
seldom  tend  to  digress.  Frequently  they  will  be  pure  plot  in  the 

way  history  sometimes  is  reported:  event  A  leads  causally  to  event  B, 
which  in  turn  leads  directly  to  event  C,  and  so  on  until  some  final 
situation  or  set  of  prevailing  conditions  is  reached. 

•  Scenarios  frequently  consist  primarily  of  visual 
diagrams  designed  to  condense  action  into  its  largest  movements. 

That  is,  they  are  intended  to  depict  long-range  views  rather  than 
highly  detailed  closeups  of  actions  and  individual  actors,  though 
the  latter  forms  are  certainly  not  precluded.  These  types  of 
scenarios  often  will  telescope  the  events  of  severcil  weeks,  months, 
or  even  years  into  a  relatively  short  space. 

•  Scenarios  are  often  deliberately  employed  to  incor¬ 
porate  and  integrate  numerous  factors  in  a  situation,  including 
military  and  political,  sociological  and  economic,  and  psychological 
and  cultural  influences  and  processes.  Thus  they  may  include  more 
frames  of  reference  than  ordinarily  are  found  in  imaginative  writing 
and  analysis. 

From  these  characteristics  certain  problems  requiring 
special  attention  by  the  analyst  become  apparent.  One  of  these  is 
to  ensure  that  at  all  stages  the  coherence  of  the  plot  or  sequence 
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of  events  is  immediately  visible:  that  an  outcome  is  believed 
possible  because  certain  earlier  events  or  conditions  are  believed 
capable  of  bringing  it  about.  In  turn,  it  should  be  clear  how  this 
outcome  influences  otl.er  stages  in  the  course  of  events  being 
described.  Again,  rationale  structures  are  being  developed. 


Table  2,  page  21,  shows  the  functional  outline  of  the  threat 
recognition  stage  of  analysis.  In  reviewing  the  table,  the  reader 
will  see  quickly  that  there  is  a  close  correspondence  between  the 
modeling  activity  —  PAMNACS,  DENs,  CFFs,  and  ANEMs  —  and  the  steps 
and  procedures  in  the  analytic  functions.  Largely  through  the 
modeling,  the  routine  is  implemented:  +he  indicators  are  matched 
against  the  various  models,  the  correlations  identified,  and  the 
models  as  a  set  compared  in  terms  of  their  relative  levels  of 
activity.  Novel  threat  analysis  is  conducted,  with  the  steps  and 
procedures  implemented  through  the  ANEW.  Hence  the  modeling  is  a 
means  of  facilitating  the  process.  Backlog  and  reiationality 
measures  enable  us  to  monitor  the  extent  of  these  activities. 


Signification.  In  the  threat  recognition  stage  the  analyst, 
according  to  a  chosen  rationale,  assigns  meaning  to  the  input  data  in 
association  with  a  series  of  models.  Having  said  this,  we  must  reflect 
on  meaning  and  context.  We  have  said  that  meaning  constitutes  a  change 
in  a  living  system's  processes  elicited  by  an  information  input.  For 
the  individual  warning  analyst  and  for  a  group  of  analysts  pursuing 


WARNING  ANALYSIS  METHODS 


the  warning  mission,  changes  that  indicate  meaning  has  occurred  are 
any  and  all  changes  in  the  perspective  on  the  likelihood  of  threat 
situations.  Input  data  must  have  more  meaning  or  less  meaning  —  but 
definitely  some  degree  of  meaning  —  in  relation  to  what  it  is  thought 
to  tell  us  with  respect  to  possible  threats  which  must  be  warned 
against.  The  information  may  appear  insignificant,  mildly  interesting, 
or  highly  indicative.  Moreover,  this  meaning  is  visible  through 
changes  in  or  to  the  contextual  models  used  in  analysis. 

The  anthropologist,  Edward  T.  Hall,  has  written  importantly 
about  the  relationship  of  context  to  meaning  Hall  explores  richly 
the  distinction  between  high  context  and  low  context  modes  of  thought 
and  expression.  High  context  modes  have  a  considerable  amount  of 
rationale  and  perspective  already  built  in  and  available  for  the 
interpretation  of  input  data  and  the  development  of  outlooks  on 
states  of  reality.  One  can  think  of  such  mode*-  as  involving  estab¬ 
lished  models  of  reality  and  systems  of  rationale  (and,  of  course, 
biases  as  well)  which  create  a  strong  context  for  the  assignment  of 
meaning  to  input  data,  all  hopefully  in  the  interests  of  perceiving 
and  interpreting  reality  for  purposes  of  developing  policies  and 
strategies  of  enlightened  self-interest  and  survival.  High  context 
systems  have  the  advantage  of  simplifying  the  cognitive  functions  of 
analysis  and  decision.  They  tend  to  ease  problems  of  information 
overload  and  stress.  The  are  intended  to  incorporate  painfully 
learned  lessons  of  experience  which  have  some  proven  reliance  as 
continuing  guidelines  for  behavior.  They  may  be  stores  of  collec¬ 
tive  wisdom  obtained  over  long  periods  and  intended  to  keep  us  from 
repeating  tragic  history.  (They  are  also  deeply  involved  with  culture 
and  cultural  identity  and  integrity. ) 

High  context  modes  of  expression  tend  formally  to  incor¬ 
porate  high  levels  of  explicit  signification,  particularly  in  terms 
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of  capturing  rationale.  A  simple  but  penetrating  example  is  given 
by  Hall  in  his  description  of  how  the  Hopi  Indians  use  their  high 
context  language  in  discussing  the  weather.  Hall  begins  by 
commenting  on  spoken  and  written  contexts: 


...  context  ...  carries  varying  proportions  of  the  meaning.. 

Without  context,  the  code  is  incomplete  since  it  encompasses 
only  part  of  the  message.-  This  should  become  clear  if  one 
remembers  that  the  spoken  language  is  an  abstraction  of  an 
event  that  happened,  might  have  happened,  or  is  being  planned. 

As  any  writer  knows,  an  event  is  usually  infinitely  more 
complex  and  rich  than  the  language  used  to  describe  it. 

Moreover,  the  writing  system  is  an  abstraction  of  the  spoken 
system  ....  in  the  process  of  abstracting,  as  contrasted  with 
measuring,  people  take  in  some  things  and  unconsciously  ignore 
others.  This  is  what  intelligence  is:,  paying  attention  to  the 
right  things.  The  linear  quality  of  a  language  inevitably 
results  in  accentuating  some  things  at  the  expense  of  others. 

Two  languages  provide  interesting  contrasts.  In  English,  when 
a  man  says,  'It  rained  last  night,1  there  is  no  way  of  knowing 
how  he  arrived  at  the  conclusion,  or  if  he  is  even  telling  the 
truth,  whereas  a  Hopi  cannot  talk  about  the  rain  at  all  without 
signifying  the  nature  of  his  relatedness  to  the  event  —  first¬ 
hand  experience,  inference,  or  hearsay. 

The  danger  in  a  high  context  system  is  entropy.  It  is  a 
poor  wager  indeed  to  suppose  that  reality  will  not  eventually  change 
in  such  ways  as  to  make  at  least  portions  of  high  context  systems 
obsolete. 


Low  context  modes  can  intensify  the  interpretive  effort, 
since  the  human  must  invent  new  context  in  order  to  assign  meaning 
and  hence  understand  and  explain  perceived  reality.  This  requirement 
brings  with  it  all  the  problems  that  high  context  modes  tend  to 
mitigate,  particularly  those  of  input  overload.  But  it  also  has  the 
advantage  of  causing  us  to  look  at  input  data  from  new  perspectives. 

It  tends  to  help  us  avoid  failing  to  recognize  novel  situations  because 
we  are  too  wedded  to  existing  context  which  may  have  suffered  entropy. 
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Obviously  the  ideal  strategy  in  warning  analysis  is  to 
achieve  the  appropriate  balance  between  high  and  low  context  analysis 
or  some  condition  <“»f  oscillation  between  the  extremes.  It  is  evident 
that  the  system  of:  models  —  PAMNACS,  DENs,  CEFs,  and  ANEMs  — 
represents  a  high-context  interpretive  mode.  The  models  establish  a 
substantial  context  within  which  to  assign  meaning  to  input  data. 

But  we  have  noted  that  the  crucial  factor  in  warning  analysis  is  the 
force  of  the  interpretive  impulse  which  causes  the  analyst  to  challenge 
the  high-context  models  when  he  cannot  assign  meaning  satisfactorily 
within  their  current  forms .  As  an  analyst  begins  any  given  sequence 
of  analysis  in  the  threat  recognition  stage,  he  may  experience  either 
of  these  cases: 

e  Case  1:  He  finds  that  the  data  may  be  read-*  ly  asso¬ 
ciated  with  a  node  in  an  existing  model. 

•  Case  2:  He  finds  that  the  data  is  not  readily  asso¬ 
ciated.  This  condition  then  requires  him  at  minimum 
to  modify  or  create  a  new  subframework  to  account  for 
the  data.  It  may  even  require  him  to  create  a  new 
model  (e.g.,  a  new  major  national  course  of  action  in 
the  PAMNACS ,  followed  by  a  new  DEN  and  associated  CEFs). 

In  Case  1,  the  primary  record  of  signification  is  held  in 
the  CEF.  Recall:  the  CEF  is  not  only  a  model  node  (and  a  carefully 
chosen  one,  as  we  discussed  in  the  earlier  comments  on  tne  art  of 
scenario  building):  the  CEF  is  also  a  kind  of  signification  refer¬ 
ence  point;  it  is  a  container  in  which  we  store  input  data  which  we 
associate  with  the  existence  of  real  activity  similar  to  that 
modeled  at  that  node.  The  analyst,  by  associating  the  data  with  that 
particular  node,  is  saying  that  one  of  the  conditions  in  a  threat 
situation  he  believes  he  has  effectively  modeled  is  occurring. 
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The  full  rationale  behind  the  assignment  cf  meaning  — 
that  is,  behind  the  signification  of  the  data  —  lies  in  the  assump¬ 
tions  and  logic  that  went  into  t.he  structure  of  the  model  itself: 
assumptions  about  the  time  frames  required  for  certain  events;  the 
preparations  needed;  and  any  and  all  other  conditions  believed  neces¬ 
sary  for  the  situation  to  occur.  Here  we  must  ask  for  the  reader's 
indulgence  temporarily.  We  cannot  deal  fully  with  the  issue  of  the 
assumptions  in  models  until  we  discuss  the  projection  stage  of 
analysis.  The  place  to  deal  with  the  assumptions  behind  models  is  in 
the  discussion  of  the  analysis  of  conditions  leading  to  threats,  and 
that  analysis  is  bound  up  in  the  process  of  making  projections. 

Although  \.a  are  describing  warning  analysis  as  a  linear  process,  it 
becomes  a  cyclical,  iterative  process  in  real  operations.  Developing 
threat  models  is  done  in  conjunction  with  other  kinds  of  analysis 
performed  in  the  projection  stage. 

In  Case  2,  the  primary  record  of  meaning  lies  both  in 
changes  made  to  the  models  and  changes  made  in  the  models.  The 
information  input,  because  it  has  not  appeared  to  have  adequate 
meaning  within  the  context  created  by  the  existing  forms  of  the 
model,  has  caused  the  analyst  to  search  for  meaning  through  the 
creation  of  new  context.  This  begins  as  low-context  analysis  whose 
thrust  is  to  create  a  new  high  context  useful  for  future  analysis. 

Machine  Support  Implications.  We  have  argued  that  special¬ 
ized  machine  support  to  the  analyst  is  essential.  We  have  said  that 
this  is  so  because  there  must  be  extrasomatic  memory  and  analysis  aids 
to  mitigate  psychophysioiogical  problems  which  constrain  human  analysis. 
Section  5  below  describes  what  we  believe  to  be  the  specific  software 
requirements  of  such  support:  the  file  architecture,  programming 
ne'is,  display  designs,  required  memory  capacities,  major  man-machine 
interface  requirements  and  other  traditional  design  considerations. 
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What  we  should  consider  in  the  present  section  are  some  important 
general  implications  about  the  basic  nature  of  such  machine  support 
to  the  warning  analyst. 

One  consideration  has  an  overriding  importance.  It  is 
speed  of  operation.  The  human  mind  can  operate  with  enormous  speed 
in  processing  and  analyzing  data.  It  may  worry  a  problem  for  long 
periods  of  time,  creating  and  reviewing  options,  imagining  conse¬ 
quences  of  action,  and  contemplating  problems  from  many  sides.  But 
as  it  does  such  analysis  --  in  the  analytical  acts  themselves  — 
it  naturally  operates  at  great  speeds .  Any  successful  extrasomatic 
support  system  must  allow  the  analyst  to  continue  to  operate  at 
adequate  speeds .  Ultimately  there  are  two  words  to  describe  the 
problems  that  otherwise  ensue:  delay  and  interruption.  If  there  is 
interruption  of  the  all-out,  speedy  processes  of  cognition  in  hard 
analysis,,  whether  the  processes  be  lightening-like  associations,  a 
series  of  associations  made  over  a  few  seconds,  or  a  general  assault 
for  a  lengthy  session  on  a  puzzling  problem,  such  as  the  significance 
of  an  anomaly,  tne  result  can  oe  devastating.  Anyone's  everyday 
experience  verifies  again  and  again  the  negative  effects  of  interrup¬ 
tion  on  difficult  analysis.  No  particular  expertise  in  human  factors 
is  needed  to  realize  that  out  of  the  many  reasons  why  analysts  may  not 
depend  on  a  given  computer-based  aid,  one  of  the  primary  ones  is  that 
it  fails  to  meet  the  need  for  sustained  analysis  at  appropriate  speeds. 
Certainly  one  of  the  diagnostic  goals  in  monitoring  the  backlog  and 
relationality  variables  is  to  discover  such  problems  anywhere  in  the 
analytic  process. 

There  is  a  serious  question  about  the  limits  in  warning 
analysis  on  the  effectiveness  of  any  approaches  using  extrasomatic 
memories  and  analytic  aids  embodied  in  books,  reports,  memos,  etc. 

It  is,  of  course,  unrealistic  to  imagine  that  we  will  cease  to  depend 
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substantially  on  these  over  the  near  term.  Nor  car  analysis  be  conduc¬ 
ted  entirely  in  a  visual  mode  using  computer-driven  graphic  displays 
primarily  comprised  of  geometric  configurations  and  alphanumerics . 

But  the  simple  fact  remains  that  books,  reports,  memos,  etc.  are 
harder  and  slower  to  use  in  virtually  all  aspects  of  analysis:  data 
review,  data  correlation,  interpolation,  integration,  etc.  Because 
of  this,  they  are  more  likely  to  suffer  entropy  in  container  drawers. 
And  because  of  these  and  other  problems,  they  will  not  provide  an 
adequate  record  of  analytic  operations  and  processes.  Just  as  a 
pianist  translates  his  ideas  into  sound  directly  through  a  keyboard 
controlling  sound-producing  mechanisms,  and  from  there  perhaps  into 
a  recorder,  so  ideally  the  analyst  should  have  a  computer-based  support 
system  that  directly  accepts,  records  and  displays  his  pertinent 
analytic  functions  and  results  at  due  speeds.  Such  capabilities  are 
vital  in  promoting  the  analysis  heuristic. 

Tolerance  of  delays  and  interruptions  is  to  some  extent 
dependent  on  the  specific  capacities  for  such  in  each  individual. 

And,  of  course,  sometimes  delays  and  interruptions  may  ironically 
prove  beneficial.  Furthermore,  certainly  the  written  word  in  paper 
storage  has  uniquely  beneficial  properties  as  a  memory  and  as  a 
vehicle  for  sophisticated  analysis.  Each  media  has  its  virtues. 

But  none  of  them  is  even  remotely  as  important  as  the  overriding 
point.  It  would  be  foolish  indeed  not  to  understand  that  speed  of 
analytic  operations  made  possible  by  computer-based  support  systems, 
together  with  the  secure  memory  these  systems  afford,  is  directly 
related  to  enormous  possibilities  of  increasingly  sophisticated  and 
realistic  human  interpretation  of  reality.  Throughout  the  present 
research  there  is  no  more  important  point.  Nor  is  there  a  clearer 
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imperative  than  in  the  fact  that  all  too  few  manhours  doing  such 
analysis  on  significant  scales  have  ever  been  expended. 

There  is  no  necessary  conflict  between  warning  analyst  and 
machine.  The  machine  not  only  need  not  constrain  the  analytic 
capability  of  the  analysts;  it  should  extend  it,  increase  the  produc¬ 
tivity  of  the  energy  of  the  human  analyst,  and  enable  more,  faster 
and  less  error-prone  analysis. 

In  terms  of  the  models  and  modeling  in  the  threat  recogni¬ 
tion  stage,  a  primary  design  goal  for  computer-baaed  support  becomes 
clear:  the  calling  up,  comparing,  and  modifying  of  models  and 
portions  of  the  models  must  be  capable  of  being  performed  quickly 
and  displayed  succinctly  and  effectively.  As  we  shall  discuss  below, 
there  is  remarkable  potential  in  the  use  of  color  displays  and  other 
devices  for  such  purposes.  The  analyst  must  be  able  to  manipulate 
the  extrasomatic  schemata  readily  and  speedily  to  sustain  the  momentum 
of  the  analytic  processes  in  his  own  mind.  In  doing  so,  he  will 
create  a  record  for  post  mortems  that  also  must  be  reviewable  with 
appropriate  speed  as  the  need  arises. 


Projection  Stage 

The  warning  analyst  now  faces  his  greatest  challenge:  the 
projection  of  situations.  As  we  have  said,  he  will  not  be  able  to 
anticipate  final  victories.  Indeed,  we  have  discussed  in  some  detail 
the  fundamental  limits  on  projections  by  human  beings. 

But  what  prospects  are  there  for  improvement  in  the  projec¬ 
tion  of  international  situations?  That  is  the  important  question. 

The  need  to  make  projections  sufficiently  trustworthy  as  to  inform 
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enlightened  policy  simply  must  continue  to  become  increasingly  a 
recognized  priority.  Yet  until  a  consider  a/:  It  effort  to  do  such 
projections  is  mounted,  an  effort  involving  open  and  intelligence 
data,  computer-based  memories  and  analytic  aids,  a  substantial 
national  community  of  analysts  and  thi:\kers  operating  with  continuity 
and  with  constant  self-review,  and  with  R&D  response  to  shortfalls, 
on  urgent  national  problems,  we  will  really  have  no  firm  basis  for 
assessing  prospects.  However,  can  there  really  be  any  acceptable 
alternative  to  the  determination  to  improve  the  national  capability? 

In  the  present  discussion,  the  intent  is  not  to  recommend 
a  single  best  way  of  approaching  warning  projections.  Obviously 
there  are  projection  methodologies  yet  to  be  invented  which  will 
better  the  current  inventory.  We  will  discuss  a  general  method 
believed  promising  as  a  basis  for  evolving  more  refined  methods.  It 
is  an  eclectic  approach,  an  admixture  of  various  methods.  There  is 
no  intent  to  provide  a  survey  of  various  projection  techniques,  nor 
is  there  an  implication  that  what  is  included  is  necessarily  more 
suitable  than,  or  implies  a  reason  for  excluding,  other  available 
techniques.  Such  qualifications  become  less  important  when  we  recall 
that  the  focus  is  not  really  on  any  current  set  of  methods  nearly  so 
much  as  on  a  man-machine  system  whereby  communities  of  analysts  may 
readily  and  heuristically  evolve  improved  warning  projection  methods. 
In  short,  the  projection  system  described  is  a  start. 

Analysis  Forms.  The  making  of  projections  presupposes  a 
set  of  forms  used  in  conjunction  with  those  in  the  threat  recogni¬ 
tion  stage.  We  can  conceive  of  cases  in  which  the  analyst,  having 
reviewed  input  data  in  the  threat  recognition  stage  and  detected 
early  signs  of  a  potential  threat  situation,  then  proceeds  to 
projecting  outcomes.  It  is  at  this  point  that  the  requirement  for 
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additional  forms  arises.  In  discussing  the&^.  forms  we  must  simultan¬ 
eously  touch  on  aspects  of  analytic  routine;  there  is  also  a  separate 
section  on  routine  per  se  following  the  discussion  of  forms. 

•  Translation  of  Threat  Models  Into  Projection  Format. 

The  first  form  is  referred  to  us  the  projection  format.  As  Thomas 
Belden  has  pointed  out,  the  warning  analyst,  whether  developing 
unimpeded  or  influenced  projections,  ultimately  has  the  obligation 
to  respond  to  the  basic  information  categories  --  who,  what,  when, 
where,  how,  and  why.  Each  presents  a  separate  analysis  challenge  or 
obligation.  At  the  same  time  they  must  ne  combined  into  warning 
judgments.  This  implies  that  a  grand  projection  methodology  must  be 
a  synthesis  of  several  submethodologies  dealing  with  various  problems 
associated  with  the  different  categories.  Similarly,  the  measures  of 
analytic  performance  must  penetrate  each  category.  In  discussing  the 
projection  stage  in  the  warning  analysis  model,  we  established  as  the 
first  procedure  the  translation  of  individual  threat  models  used  in 
the  threat  recognition  stage  into  a  projection  format.  The  format, 
first  adopted  by  Belden,  is  this: 

(1)  Who  (or  what) 

(2)  (Could  do)  what 

(3)  (To)  whom  (or  what) 

(4 )  Where 

(5)  How 

(6)  When 

(7)  Why  (both  to  what  end  and/or  because  x  conditions  apply). 


Of  course  this  form  applies  whether  we  are  considering  a  single  state¬ 
ment  of  a  threat  or  an  extended  description  of  one. 
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The  information  categories,  combined  into  the  English 
sentence,  form  a  penetrating  and  elegant  way  to  structure  the  problem 
of  warning  projections.  The  proper  focus  on  warning  projections 
should  be  on  the  essential  output  to  decisionmakers  and  other  users; 
and,  as  simply  and  directly  as  possible,  on  each  of  the  major  challenge 
areas  to  the  analyst  in  developing  that  output.  It  is  useful  to 
consider  briefly  the  obligations  of  the  warning  analyst  in  developing 
projections  within  each  information  category. 

First  Analysis  Obligation  —  Who  and  What.  W?io  and  what  are 
explicitly  contained  in  the  sentences  formulating  the  basic  warning 
concern: 

Who  might  do  what  to  whom?  and 

What  might  cause  what  to  happen  to  whom? 

For  analysts,  the  problem  of  who  and  what  referring  to 
initiators  or  catalysts  of  threat  enactment  essentially  reduces  to 
identifying  countries,  political  and  military  groups,  leaders/ 
decision  makers,  and  economic,  psychological,  and  cultural  forces 
which  can  enact  threats  or  cause  threats  to  be  enacted. 

The  problem  of  what  as  denoting  a  hostile  action  — 
nationalize  industries,  attack  property  and  citizens,  etc.  —  and 
the  problem  of  whom  as  the  victim  of  the  hostile  action,  are  closely 
related.  Any  generalized  list  of  types  of  U.S.  interests  and  assets 
and  the  potential  threats  toward  them  would  include  items  such  as 
these : 

Attacks  against  U.S.  military/government 
personnel,  facilities,  and  equipment  located 
outside  the  U.S. 
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Hostile  actions  directed  against  other  U.S. 
citizens  located  outside  the  U.S. 

Seizure  of  U.S.  overseas  commercial  investments. 

Denial  to  U.S.  of  use  of  lines  of  communications. 

Withdrawal  of  political  support  in  world/regional 
organizations  for  U.S. 

Economic  activity  directed  against  the  U.S.,  such  as 
formation  of  cartels  to  deny  combinations  of  vital 
resources  to  U.S. 

Attacks  against  U.S.  treaty  allies. 

As  noted,  a  major  (but  not  exclusive)  principle  implicit 
here  is  that  the  proper  focus  of  warning  analysis  is  primarily  on 
threats  to  U.S.  interests  and  assets.  Identifying  potential  initia¬ 
tors  and  catalysts  of  threats,  and  identifying  and  prioritizing  the 
U.S.  assets  and  interests  threatened,  can  be  considered  the  first 
obligation  of  the  warning  analyst. 

Second  Analysis  Obligation  —  Why.  The  second  issue 
incumbent  on  the  analyst  is:  Why.  This  can  be  stated  as: 

Who  might  do  what  to  whom  because  - 


This  need  to  address  why  is  perhaps  the  most  fundamental  and  certainly 
the  most  challenging  obligation  of  warning  analysts.  The  overall 
problem  of  credibility,  including  credibility  with  decision  makers. 
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obviously  is  linked  to  establishing  the  plausibility  of  potential 
threats.  In  the  most  cosmic  sense,  theories  of  what  determines  the 
course  of  events  range  from  the  view  of  Carlyle  that  the  activities 
of  leaders  are  the  predominant  factor;  to  the  outlook  of  Tolstoy 
which  views  history  as  an  accumulation  of  accidental  events  with 
the  element  of  chance  more  important  in  determining  its  course  than 
the  decisions  of  leaders.  Both  theoretical  positions  happen  to  be 
useful;  either  may  dominate,  depending  on  the  situation.  More 
important  to  the  problem  of  method  and  procedure  is  that  the  analyst 
must  have  succinct,  up-to-date  information  based  on: 

The  goals,  predispositions  and  characteristic  behavior 
patterns  of  key  decision  makers 

Decisive  military,  economic,  political,  and  cultural 
forces . 

Further  these  must  be  tied  in  with  threats  to  U.S.  interests  and 
assets . 


Third  Analysis  obligation  —  When,  How,  and  Where.  The 
third  obligation  is  to  consider  when,  how,  and  where  a  threat  might 
be  enacted.  Each  of  these  presents  a  different  challenge  to  analysts 

For  the  problem  of  when ,  it  is  necessary  to  attempt  to 
take  into  account  all  the  necessary  processes  that  must  occur.  It 
was  noted  that  the  warning  analyst  can  focus  on  a  finite  (and,  of 
course,  prioritized)  set  of  threats.  For  the  problem  of  when,  it 
is  important  that  the  analyst  attempt  to  identify  the  essential 
processes  —  military,  political,  economic,  psychological  —  that 
must  unfold  in  the  enactment’  of  given  threats,  and  then  estimate 
the  minimum  or  worst  case  as  well  as  other  potential  timelines 
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involved  in  the  unfolding  of  these  processes.  Important  items  in 
threat  enactment  could  include  such  factors  as  the  extent  of  emer¬ 
gency  powers  of  a  government,  its  command  and  control  capabilities, 
terrain,  and  the  nature  of  U.S.  assets  threatened. 

The  problem  of  how  requires  that  the  analyst  consider  tne 
means  of  threat  enactment  at  the  disposal  of  foreign  decision  makers 
and  the  economic,  political,  or  other  forces  that  might  combine  to 
produce  enactment  of  a  threat.  The  objective  is  to  estimate  the 
relative  likelihoods  of  occurrence  of  plausible  processes  of  threat 
enactment.  Factors  such  as  the  size  and  nature  of  police/military 
forces  of  the  hostile  government  and  the  political  constraints  on  that 
government  would  be  significant. 

As  regards  where  —  the  problem  of  projecting  the  geography 
of  threat  enactment  •—  the  geography  of  U.S.  assets  and  interests 
clearly  is  a  key  factor.  In  varying  degrees,  political,  military, 
and  environmental  factors  will  also  be  important. 

Table  5  provides  a  very  simplified  and  hypothetical  example 
of  the  translation  of  a  threat  model  into  the  predictive  format.  Such 
formats  could  be  used  for  all  threat  models  and  both  types  of  projec¬ 
tion,  and  could  accommodate  considerably  more  detail  and  specificity. 

•  Specificity  Ratings.  A  prototype  Specificity  Rating 
Scale  has  been  developed  for  use  in  comparing  the  specificities  of 
models,  predictions,  and  forecasts  across  the  information  categories. 
Table  6  exemplifies  the  form,  showing  the  thrust  of  the  approach.  Of 
course,  refinements  would  be  developed  through  operational  use  of  the 
scale.  In  computer-based  operations,  color  displays  could  be  used, 
and  specificity  levels  could  be  color-coded.  In  turn,  models  used 
in  threat  recognition  and  projection  (see  below  for  examples  of  the 
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Example  Use  of  Information  Categories  With  Threat  Models 


1.  WHO  (or  WHAT):  North  Korean  combat  forces 

2.  (Could  do)  WHAT:  could  be  directed  to  launch 
surprise  attack  from  present  positions 

3.  (to)  WHOM  (or  WHAT):  against  the  ROK,  and  the 
U.S.  military  forces  stationed  in  ROK 

4.  WHERE:  by  attacking  across  the  DMZ  and  into 
ROK  via  various  routes 

5.  HOW:  by  attacking  without  mobilization  or 
forward  movement  of  reserve  forces  prior  to 
attack,  and  by  using  units  now  deployed  near 
the  DMZ;  by  flying  limited  interdiction  strikes 
by  NKAF  without  a  major  forward  deployment  of 
aircraft;  and  by  conducting  limited  naval  opera¬ 
tions  shortly  after  ground  forces  cross  the  DMZ 

6.  WHEN:  within  one  week 

7.  WHY  (or  because  X  conditions  apply) 

a.  Assume  North  Korea's  long-range  goal 
remains  the  unification  of  the  entire 
Korean  Peninsula  under  the  North's 
regime 

b.  North  Korea  perceives  an  anarchic  situa¬ 
tion  developing  in  ROK  via  the  following: 
uncontrolled  student  demonstrations;  a 
coup  d'etat;  an  economic  collapse;  an  ROK 
government  leadership  crisis. 

c.  North  Korea  perceives  that  the  ROK  is 
losing  its  ability  to  offer  determined 
resistance 
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latter)  could  be  displayed  with  use  of  the  color  codes  as  a  means 
of  rapidly  and  effectively  indicating  comparative  specificities  for 
different  elemen  ;s  in  their  structure.  Experimentation  by  the  project 
team  with  such  approaches  using  a  Tektronix  4027  color  display  have 
shown  considerable  promise. 

•  Recourse  to  Basic  Principles  Underlying  Projections. 

We  have  discussed  the  imperfection  of  projections  made  by  human 
beings.  Yet  in  his  obligation  to  make  projections,  the  warning 
analyst  must  assume  that  there  are  sc-me  reliable  rules  or  principles 
of  change  to  which  he  has  recourse  in  attempting  to  understand  the 
future  of  social  systems,  their  processes,  technologies,  behavior 
patterns  and  so  on.  In  his  very  useful  book,  An  Incomplete  Guide  to 
the  Future,  Willis  Harman  has  formulated  a  wise  approach  to  making 
piojections.  Harman's  focus  is  not  warning  as  we  are  examining  it, 
bi.t  portions  of  his  thinking  apply  here  as  well.  Harman  reminds  us 
that  although  there  are  few  clear  cut  ways  in  which  the  behavior  of 
social  systems  are  dependable,  one  may  usefully  distinguish  several 
general  principles  to  consider  when  concerned  with  the  dynamics  of 
complex  systems.  These  are  continuity,  self-consistency,  similarities 
among  systems ,  cause-effect  relationships,  holistic  trending  and 
goal-seeking.  Ultimately  the  various  methods  of  projection,  from  the 
highly  qualitative  to  the  essentially  quantitative,  are  based  on 
various  mixes  of  these  principles.  It  is  our  contention  that  these 
principles  should  be  considered  systematically  by  warning  analysts 
maxing  projections. 

What,  specifically,  do  the  principles  encompass?  Here  are 
brief  definitions  largely  adopted  from  Harman: 

Continuity-  Large  systems  show  continuity.  They  tend 
to.  flow  into  new  states  rather  than  change  in  discontinuous  jumps. 
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Potentially  a  serious  trap  in  threat  projections  is  not  to  realize 
that  even  in  civil  war  and  other  conflict  situations,  substantial 
portions  of  the  class  structure,  bureaucratic  structure,  economy 
and  technology  of  a  nation  may  survive.  In  making  projections,  we 
instinctively  and  rightly  operate  on  the  basis  of  past  experience, 
The  principle  of  continuity  may  be  invoked  in  various  projections 
involving  economic  patterns,  military  capabilities  and  postures, 
etc. 


Self-Consistency.  A  second  guiding  principle  is  that  such 
systems  tend  to  be  internally  self-consistent.  For  example,  the 
behavior  of  one  social  sector  typically  is  not  inconsistent  with  that 
of  another.  Large  scale  industries  xn  consumer  luxury  items,  for 
instance,  are  unlikely  to  develop  in  a  small,  relatively  poor  nation 
focused  on  a  military  buildup.  This  principle  is  fundamental  to  the 
scenario  technique.  As  Harman  notes,  the  purpose  of  developing 
scenarios  about  the  future  is  to  make  certain  that  the  future  condi¬ 
tions  we  postulate,  no  matter  what  technique  of  forecasting  underlies 
them,  are  systematically  examined  for  coherency  and  plausibility. 

Similarities  Among  Systems.  Systems  in  different  places 
and  times  will  show  definite  similarities.  For  example,  military 
juntas  will  tend  to  act  somewhat  alike  under  similar  circumstances. 
Harman  notes  that  this  principle  is  used  in  what  he  calls 
" anthropological"  approaches  to  studying  the  future  and  in  cross- 
cultural  comparisons. 

Cause-Effect  Relationships.  The  dynamics  of  nations, 
organizations,  and  groups  may  be  partially  understood  in  terms  of 
cause  and  effect  relationships.  Assumptions  of  cause  and  effect 
relationships  are  made,  for  example,  in  analyzing  economic  dynamics: 
if  certain  goods  become  less  available,  it  is  often  assumed  that  the 
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price  of  these  goods  will  go  up.  This  principle  is  basic  to  what 
Harman  calls  "cross  impact  analysis,"  a  widely  used  method  of  studying 
the  future  by  modeling  the  imagined  cross  impact  of  various  events 
on  one  another. 

Holistic  Trending.  In  evolving  and  changing,  systems 
'•behave  like  integrated  organic  wholes.  They  have  to  be  perceived  in 
their  entirety."  There  is,  therefore,  no  substitute  for  human  obser¬ 
vation  and  judgment  about  the  future  state  of  a  system.  Ultimately, 
no  matter  what  the  inventory  of  machine-based  projection  algorithms, 
signification  must  involve  the  human  analyst.. 

Goal  Seeking.  Like  individuals,  systems  such  as  nations, 
organizations,  and  groups  have  goals.  These  goals  may  be  more  or 
less  "conscious"  or  enunciated.  (Certainly  an  analyst  may  observe 
a  system  and  come  to  believe  that  the  real  goals  are  not  the 
announced  goals.-;  But  the  major  point,  as  Harman  states,  is  that 
"change  is  not  aimless,  however  obscured  the  goal." 

•  U.^e  of  Forms  Which  Systematically  Employ  the 
Principles  of  Projection  to  Reduce  Bias.  The  key  concept  is  that  the 
six  principles  snould  be  used  systematically.  Projection  methodolo¬ 
gies  should  be  designed  to  foster  the  application  of  analytic  energies 
in  systematic  cor  rider ations  of  possible  change.  There  are  a  number 
of  potential  approaches  using  the  principles.  These  may  primarily 
involve  quantitative  data  on  economic  and  political  indicators:  and 
hard  data  on  such  changes  as  military  technology  developments.  Or 
methods  might  be  used  which  attempt  to  project  qualitative  aspects 
such  as  shifts  in  cultural  perspectives  (such  as  the  resurgence  of 
traditional  values).  Methods  combining  these  separate  approaches 
might  be  used  which  ultimately  are  primarily  holistic. 
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The  difficulty,  of  course,  lies  in  avoiding  bias.  We  have 
stressed  the  difficulty  in  gaining  objective  views  of  the  future. 

As  we  noted,  the  analyst  runs  a  high  risk  of  bias  when  he  relies  too 
heavily  on  one  projection  technique.  As  Harman  points  out,  methods 
that  rely  essentially  on  numerical  data  are  subject  to  bias  toward 
input  information  that  can  be  readily  quantified.  Similarly,  method¬ 
ologies  which  focus  narrowly  on  theoretical  issues  and  rational 
behavior  may  lead  to  analytic  failures  because  irrational  and  uncon¬ 
scious  forces  are  insufficiently  accounted  for. 

An  obvious  strategy  of  the  warning  analyst  in  mitigating 
bias  is  to  use  several  approaches  to  projection.  The  problem  will 
lie  in  judging  which  results  to  stress  in  the  various  approaches  and 
which  to  downgrade  in  importance  for  given  analysis  problems.  One 
tactic  is  to  look  for  points  of  similarity  and  conflict  in  the 
results  and  pursue  the  latter  as  needed.  More  basically,  the  warning 
analyst  would  be  prudent  to  consider  several  plausible  future  situa¬ 
tions,  not  a  single  one,  given  the  perennial  uncertainty  about  the 
future.  In  the  discussion  of  threat  recognition,  this  strategy  «as 
apparent  in  the  construction  of  models  of  alternative  threats.  The 
principles  of  forecasting  can  be  used  in  a  variety  of  ways  to  delineate 
a  set  of  alternative  paths  to  the  future.  In  the  authors  judgment, 
little  has  been  done  to  develop  analytic  forms  which  deliberately  use 
all  the  principles.  A  few  promising  ones,  some  described  by  Harman 
and  some  developed  in  the  present  research,  and  all  readily  compatible 
with  computer  and  computer-driven  display  operations,  are  as  follows: 

Sector  Comparison.  Described  by  Harman,  this  form  can  be 
used  with  the  threat  models  and  with  several  projection  techniques 
discussed  below.  Sector  comparison  is  an  analytic  technique  for 
examining  the  plausibility  of  various  projected  situations,  given 
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differing  combinations  of  military,  economic,  technological,  political, 
and  other  conditions.  The  warning  analyst  would  apply  the  sector 
comparison  technique  as  follows: 

1)  In  the  case  of  North  Korea,  for  instance,  consider  a 
number  of  sectors  of  that  nation,  such  as: 

E  =  economic  performance 
S  =  economic  structure 
M  =  military  strength 
R  =  military  readiness 
B  =  outlook  of  decision  makers 
A  =  relations  with  key  nations 

I  =  political  control  exercised  by  current  regime 

2)  For  each  sector,  define  alternative  conditions  that 
together  span  a  reasonable  rarge  of  possibilities. 

The  range  of  possibilities  may  be  as  extensive  as  the 
analyst  judges  useful.  Table  7  gives  simplified 
examples .  The  question  mark  over  the  last  column 
indicates  that  continuous  review  and  search  for  addi¬ 
tional  categories  of  importance  must  occur. 

3)  Examine  plausible  sector  combinations  such  as 

E2  -  etc.  Use  projection  principles  such  as 

internal  self  consistency,  holistic  trending,  cause 
and  effect,  etc.,  to  analyze  plausibilities  among 
these  sector  combinations . 


■Mil 


L  *>■  - 
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Table  7.  Example  Sector  Segmentation 


E^  Widespread 
prosperity 

E2  Prosperity 
restricted 
to  certain 
groups 

E3  Equilibrium, 
slow  growth, 
general 
public 

satisfaction 

E4  Recession, 
general  dis¬ 
satisfaction 

E5  Economic 
depression 


B 

Near-term 
takeover  of 
ROK/ expul¬ 
sion  of  UN 
forces  is 
considered 
plausible 

B2  Expectation 
of  takeover 
of  ROK  by 
mid  1980' s 

B,  Acceptance 
of  ROK  as 
reality  over 
indefinite 
future 
period 

B^  Focus  on 
internal 
North  Korean 
economic  and 
political 
problems 

B,.  Non-align¬ 
ment  stance 


M 

Three-to-one 
advantage  on 
ground  over 
forces  in 
ROK;  AF 
superiority 

M,,  Super- ority 
in  AF;  rough 
equality  in 
ground 
forces 

M3  Inferiority 
in  forces 
but  good 
defenses 

Questionable 

defense 

Mili  tary 
forces  in 
poor  status 


1^  Current 
regime 
firmly  in 
control 

I2  Current 
regime 
in  control 
of  most  of 
country 

I  Current 
regime 
in  control 
of  key  urban 
centers  only 

14  Current 
regime  in 
control  of 
capital  and 
few  other 
areas 

15  Current 
regime 
extremely 
vulnerable 
to  opposi¬ 
tion 
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futures  mapping.  The  approach  is  based  on  the  fundamental  concept 
that  over  a  specified  period  of  time  there  is  a  definite  range  over 
which  sector  combinations  can  vary  Per  example,  it  is  unlikely  that 
a  nation  would  go  from  a  classic  depression  to  substantial  recovery 
in  one  year.  The  probable  futures  mapping  would  be  done  in  the 
following  steps. 


1) 


Segmentation  of  Sectors.  The  types  of  sectors  described 
earlier  should  be  analyzed  and  segmented  into  discrete 
states.  These  segments  must  be  fully  described  to 
reduce  the  ambiguity  between  levels.  The  process  will 


be  continuously  refined  as  the  emalyst  gains  experience 


and  insight  into  its  use  in  a  given  problem  area. 


2 )  Detailed  Probable  Futures  Mapping  for  Each  Sector. 
Beginning  with  a  description  cf  the  present  in  terms 
of  the  sectors  defined  in  Step  1,  the  analyst  then 
extrapolates  into  the  future  and  analyzes,  for  each 
sector  and  for  every  time  period,  which  segments  are 
probable.  Typical  single  sector  mappings  are  shown 
in  Figures  13-16  with  respect  to  military  (M) ,  economic 
( E ) ,  internal  security  (I),  and  leadership  outlook  {B) 
sectors.  The  analyst  has  decided  that  over  specific 
periods  of  time  the  military,  economic,  internal 
security,  and  leadership  outlook  conditions  could 
c range  according  to  various  and  varying  schedules. 


3 )  S/nthesls  of  Single-Sector  Mapping.  The  maps 

developed  in  Step  2  are  now  combined  into  a  probable 
futures  map,  as  exemplified  in  Figure  17.  Beginning 
with  the  present  (the  center  of  the  concentric  circles 
of  time)  possible  combinations  of  the  sectors  are 


F:.gure  13.  Single-Sector  Map  —  Military 


Figure  34. 


Single-Sector  Map  —  Econorai 
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Figure  15. 


Single-Sector  Map  --  Leadership-Outlook 
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Figure  17.  Example  Probable  Futures  Map 
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displayed  along  each  time  circle.  The  time  frames  used 
for  the  purpose  of  this  example  are  1  month,  1  year, 

2  years  and  5  years.  The  results  of  Step  2  are  overlaid 
onto  the  futures  map  to  achieve  a  probable  futures  map. 
Thus  the  analyst  has  decided  on  a  definite  range  of 
possible  conditions  over  several  time  increments.  This 
technique  is  intended  to  help  the  analyst  focus  on  the 
probable  future  and  not  dilute  his  attention  on  the 
vast  range  of  possibilities.  The  arrows  on  the  map 
indicate  unique  conditions  for  given  combinations .  As 
we  shall  discuss  below,  such  prompting  aids  as  the 
arrows  anticipate  a  set  of  computer-based  graphic 
displays. 

4)  Sector  Analysis.  To  this  point  we  have  been  concerned 
with  possible  conditions,  but  not  explicitly  with 
possible  threats.  The  analyst  now  uses  the  results  of 
the  sector  comparison  technique  to  review  all  the 
threats  of  interest.  The  result  of  this  step  is  the 
identification  of  a  time  frame  and  sector  combination 
associated  with  each  threat.  This  step  connects  the 
analysis  of  possible  and  probable  conditions  to  the 
threat  modeling  performed  previously  in  the  threat 
recognition  stage. 

5)  Identification  of  "Hidden11  Threats.  This  brings  us  to 
a  very  important  point.  The  mapping  which  results  from 
Step  4  will  almost  certainly  display  various  probable 
futures  described  by  unique  sector  combinations  not  all 
of  which  have  a  previously  imagined  threat  associated 
with  them.  The  analyst  must  study  each  of  these 
probable  sector  combinations  and  either  identify  a  new 
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threat  which  falls  into  that  sector  but  had  not  been 
analyzed  before;  or  satisfy  himself  that  no  threat 
exists  for  that  sector  combination.  This  step  aids  the 
analyst  in  discovering  previously  undefined  threats  that 
are  within  the  scope  of  a  probable  future. 

6)  Plotting  the  Unimpeded  Threat  Line.  With  the  probable 
futures  map  completed,  and  possible  threats  located, 
the  analyst  has  the  task  of  plotting  the  unimpeded 
threat  line  —  the  path  into  the  future  that  the 
subject  country  will  most  likely  take,  given  no  change 
in  our  present  day  policies  and  other  current 
constraints . 

7)  Assessing  Impact  on  Unimpeded  Threat  Line.  Finally, 
given  the  unimpeded  threat  line  is  not  leading  to 
desirable  situations,  the  probable  futures  mapping 
technique  can  serve  as  a  framework  in  reviewing  how 
changes  may  be  instigated  to  alter  the  unimpeded 
threat  line  (and  thereby  create  an  influenced  projec¬ 
tion)  . 

Divergence  Mapping.  Discussed  by  Harman,  divergence 
mapping  is  a  form  of  analysis  which  can  be  used  for  analyzing  and 
mapping  a  number  of  possible  threat  situations  in  terms  of  their 
comparative  likelihoods.  It  maintains  the  thrust  of  analysis  that 
develops  in  the  sector  comparison  and  probable  futures  mapping 
approaches.  Development  of  divergence  maps  involves  the  following 
steps : 
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1)  Review  the  basic  principles  underlying  forecasting, 
such  as  continuity,  internal  self-consistency,  simi¬ 
larities  among  systems,  cause-effect  relationships, 
etc . 

2)  Consider  each  threat  modeled  in  the  threat  recognition 
stage  as  a  potential  situation  in  the  future.  Also 
define  non- threatening  situations. 

3)  Title  each  situation  and  place  it  within  a  block, 
then  graphically  arrange  the  blocks  in  a  plausible 
seguence  on  a  "map,"  as  follows: 

(a)  Identify  those  situations  most  like  the  present, 
based  on  systematic  recourse  to  forecasting 
principles. 

(b)  For  methodological  purposes,  initially  consider 
these  situations  to  be  located  nearest  in  time 
to  the  present. 

(c)  Identify  the  situations  least  like  the  present 
and  initially  locate  them  furthest  from  the 
present. 

4)  Complete  the  divergence  map  by  filling  in  the  inter¬ 
mediate  situations.  Simplified  examples  used  to 
show  the  approach  are  depicted  in  Figure  18. 

(a)  Extending  out  over  time,  there  are  several  tiers 
of  situations.  There  could  be  more,  of  course, 
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(b)  Summarizes  the  most  plausible  paths  and  the 
associated  rationale. 

(c)  Assigns  a  time  scale  to  each. 

(d)  Defines  the  least  plausible  paths. 

Employment,  of  General  and  Opposing  Projection  Methodologies . 
In  order  to  help  mitigate  bias  problems  and  consider  potential  situa¬ 
tions  in  a  realistic  fashion,  the  warning  analyst  can  employ  two 
general  and  opposing  projection  methodologies  which  can  be  described 
as  present  extrapolation  and  future-backwards .  These  can  be  used  with 
the  other  techniques  described  above. 

1)  The  present  extrapolation  method  of  projection  extends 
events  from  the  present  conditions  to  future  situa¬ 
tions.  The  description  of  the  time-phased  events  that 
comprise  these  extrapolations  are  very  specific  in 
their  details  in  time  periods  close  to  the  present, 
becoming  progressively  more  general  and  vague  further 
our  in  time. 

2 )  The  future -backwards  method  begins  with  a  precise 
description  of  a  future  situation,  for  example,  a 
specific  threat.  The  analyst  then  works  backwards 
in  time  via  what  he  considers  necessary  paths  that 
uncover  required  pre-conditions  for  the  future  situa¬ 
tion.  Use  of  the  two  methods  together  can  be  valuable 
since  logical  discontinuities  in  a  projection  may 
become  more  apparent  through  a  comparison  of  the  two 
projections 
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Modeling  of  Interactive  Processes.  Especially  in  making 
influenced  predictioi  s  and  forecasts,  the  analyst  becomes  concerned 
with  the  decisive  interactions  among  various  systems  and  forces  in  a 
situat.on.  The  basic  concept  is  that  the  entities  in  crisis  situa¬ 
tions  will  interact;  they  will  respond  to  each  others'  actions. 

Thomas  Belden  has  modeled  this  phenomenon  as  a  decision  stairway 
expressed  in  terms  of  time  (horizontal  axis)  and  probability  (vertical 
axis).  To  take  a  simple  case,  let  us  suppose  that  one  force  is  intent 
on  launching  a  surprise  attack  on  another.  The  attacker  must  perform 
variou ,  preparatory  logistical,  reconnaissance,  force  mobilization, 
and  ot -ier  acts  before  he  is  ready  to  attack.  One  can  think  of  these 
acts  as  forming  a  sequence  leading  up  a  staircase  toward  increasing 
readir, sss  to  attack  and,  ultimately,  the  decision  to  do  so.  If  at 
some  p-'.int  the  potential  victim  detects  such  preparations  and  reacts 
on  that  basis  (for  example,  signals  awareness),  the  attacker  may  be 
deterred  or  halted  and  proceed  back  down  the  stairway  and  away  from 
readiness  to  attack  and  the  actual  decision  to  do  so.  In  making 
projections,  the  analyst  must  be  attentive  to  the  critical  impact  of 
such  interactive  processes  on  likely  outcomes. 

Rationale  Recording.  What  we  have  been  describing  are  some 
recommended  forms  of  warning  projections.  We  have  developed  a  general 
fraiaework  as  follows- 

Projections  ultimately  channel  into  the  basic  infor¬ 
mation  categories 

The  state  of  social  systems  and  other  entities  may  be 
considered  in  terms  of  the  status  of  certain  of  their 
sectors.  Various  states  of  these  sectors  are  more  or 
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less  likely  in  various  combinations.  These  combinations 
essentially  amount  to  sets  of  conditions. 

The  likelihood  of  given  sets  of  conditions  at  various 
future  points  should  be  estimated  with  recourse  to 
several  basic  principles  affecting  societal  dynamics. 

Possible  threats  should  be  "overlaid"  onto  the  mappings 
of  possible  conditions. 

The  mappings  should  ba  used  as  an  aid  in  identifying 
threats  not  previously  considered. 

The  mappings  should  also  be  used  as  an  aid  in  postu¬ 
lating  the  most  likely  unimpeded  conditions  out  to 
various  future  points  and  the  implications  for  U.S. 
policy. 

But  the  question  remains,  what  about  the  specific  rationales 
behind  the  projections?  Exactly  what  rationales  can  analysts  use  in 
systematically  worrying  projection  from  the  point  of  view  of  the  basic 
principles  of  societal  dynamics?  How  can  they  resolve  the  inevitable 
tensions  among  the  competing  rationales? 

There  are  a  number  of  approaches,  ranging  from  qualitative 
to  quantitative,  and  sometimes  involving  a  blend  of  the  two.  Models 
of  societal  change  and  international  conflict  are  constructed  on  the 
basis  of  paradigms  of  psychology  and  sociology.  As  another  approach, 
argument  by  historical  analogy  is  made;  for  example,  detailed  models 
of  oast  crises  and  military  surprise  are  constructed  as  a  basis  for 
recognizing  future  threats.  In  still  another  approach,  Bayesian, 
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Markovian,  and  other  methodologies  based  on  statistical  approaches 
involving  probability  have  been  developed. 

There  is,  however,  little  doubt  that  we  are  in  Idle  infancy 
of  the  development  of  warning  projection  techniques,  a  point  directly 
or  indirectly  implied  at  every  point  i i  the  present  research.  As 
just  one  example,  the  critical  need  in  warning  analysis  to  understand 
better  the  psychophysiological  constraints  on  foreign  leadership 
behavior,  especially  in  a  predictive  sense,  is  largely  unmet.  There 
is  no  doubt  that  this  is  a  huge  problem  with  substantial  political 
overtones,  but  progress  has  been  far  too  limited.  In  a  report 
prepared  for  the  Defense  Advanced  Research  Projects  Agency,  Gerald 
W.  Hopple  states: 

...  the  elucidation,  measurement,  and  analysis  of  an  array  of 
individual-  and  elite-level  factors  should  enhance  our  capa¬ 
bilities  for  ....  estimating  and  predicting  the  intentions, 
preferences,  and  probable  behaviors  of  other  actors  in  the 
international  arena .: 

But  Hopple  must  then  note  that  although  there  has  been  considerable 
research  in  the  various  relevant  fields,  the  following  is  reality: 

As  is  customaiy  in  social  scientific  inquiry,  however,  the 
existing  work  is  disparate,  uneven,  and  ad  hoc  in  nature. 

Few  efforts  have  been  undertaken  to  map  out  the  terrain  in 
more  than  a  cursory  fashion. 

It  is  not  our  purpose  here  to  survey  and  compare  various 
specific  projection  techniques  currently  available;  there  is  consi¬ 
derable  literature  on  these  techniques.  From  the  present  perspective, 
what  we  do  say  about  them  is  two  things:  first,  they  should  fit 
usefully  into  the  basic  framework  of  warning  projections:  and  second, 
they  should  be  considered  as  heuristic.  For  we  may  be  certain  that 
there  is  now  nothing  like  a  fully  satisfactory  projection  technique 
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for  the  warning  problem.  Improved  techniques  must  be  developed  over 
time.  As  is  being  argued  in  these  pages,  the  key  to  such  development 
lies  in  extrasomatic  memories  and  analytic  aids. 

Regardless,  we  are  concerned  with  the  capturing  of  whatever 
rationale  is  used  by  warning  analysts  making  projections,  no  matter 
how  qualitative  or  quantitative,  rudimentary  or  sophisticated;  for 
rationale  must  be  recorded  for  purposes  of  measuring  signification 
(and  ultimately  for  operations  research  in  post  mortems  to  identify 
technological  and  methodological  needs). 

We  may  consider  rationale  recording  at  two  levels:  the 
level  of  conditions  and  the  level  of  threats.  The  formats  of  the 
rationale  records  should  not  differ  significantly  between  the  two. 

At  the  first  level  the  analyst  is  recording  the  rationale  behind 
the  conditions  he  has  projected  will  hold  in  a  given  country  or 
sphere  of  interest  at  a  given  time.  Table  8  is  a  matrix  relating 
rationale  to  the  projected  conditions  and  the  principles  of  change. 

The  matrix,  which  is  not  in  a  completed  status,  is  intended  as  a 
simplified  example.  The  statements  of  rationale  can  be  as  technical 
and  detailed  as  warranted  and  the  analyst  is  capable  of  producing. 

In  a  computer-based  memory  system  with  graphics  displays,  obviously 
there  must  be  abbreviated  rationale  statements  for  quick  display  and 
review,  with  access  to  more  detailed  back-up  statements,  some  of  which 
may  not  need  to  be  computer  stored  (for  example,  "fixed"  rationales 
such  as  those  in  Bayesian  approaches). 

Useful  supporting  statements  concerning  the  basic  assump¬ 
tions  behind  bot.i  levels  of  rationale  recording  *-  conditions  and 
threats  —  should  be  employed.  Very  simple  formats  for  such  struc¬ 
tures  are  provided  in  Table  9,  Panels  A  and  B.  (While  the  formats 
are  simple,  the  accompanying  statements  of  rationale  can  be  detailed 
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and  substantial.)  Jerry  Kidd  has  suggested  a  vector  approach  (see 
Panel  B)  in  which  the  analyst,  by  whatever  rationale  and  methods  are 
available  to  him,  develops  vectors  whose  length  is  a  measure  of  the 
relative  impact  of  the  various  (sometimes  competing)  principles  of 
change  and  then  resolves  the  vectors  and  formulates  a  projection. 


To  suggest  how  new  projection  rationales  can  be  adopted 
for  warning  analysis,  consider  a  promising  new  approach  to  projec¬ 
ting  societal  dynamics  by  the  systems  theorist,  John  Sutherland. 
Sutherland  summarises  his  paper,  A  Syncretic  Inquiry  into  Societal 
Dynamics ,  as  follows: 

The  popular  paradigms  of  modern  psychology  —  biologism,  behav¬ 
iorism  and  humanism  —  have  been  used  to  generate  constructs  of 
macrosociological  significance.  But  a  simple  one-to-one  mapping 
from  the  psychological  to  the  macrosociological  domain  restricts 
the  richness  and  variety  exhibited  in  the  latter.  A  somewhat 
more  complete  mapping  process  is  perhaps  required  .-.•>>  Particu¬ 
larly,  we  are  concerned  with  the  procedures  and  implications  of 
syncretic  model-building.  This  process  appears  to  have  some 
advantages  when  the  discipline(s)  at  hand  are  partitioned  into 
competitive  or  recriminatory  theoretical  camps  (as  are  pschology 
and  macrosociology) .  The  basic  strategy  is  to  raise  the  tradi¬ 
tional  units  of  analysis  to  some  higher  level  of  abstraction, 
such  that  otherwise  obscured  points  of  complementarity  may  become 
apparent.  Syncretic  models  are  thus  analytical  hybrids  -- 
neither  normative  nor  descriptive  —  and  thus  impose  essentially 
unique  challenges  to  their  architects.  One  way  in  which  these 
challenges  may  be  met  is  (summarized  thusly) . 

Particularly,  the  three  dominant  paradigms  of  psychology  are 
used  to  generate  a  set  of  collective  behavioral  referents  that, 
in  concert,  are  suggested  to  pretty  well  exhaust  the  basic 
referents  required  by  macrosociology.  The  heuristic  thesis 
underlying  the  mapping  process  is  this:  at  the  macrosociological 
level  of  inquiry  there  is  a  strong  clustering  effect,  such  that 
certain  behavioral,  social,  economic  and  political  attributes 
will  tend  to  appear  in  concert  (i.e.  ,  be  highly  intercorrelated) . 
That  is,  certain  admixtures  of  behavioral,  social,  economic  and 
political  properties  are  distinctly  more  probable  than  others. 

It  is  the  most  dominant  of  these  admixtures  that  serve  to 
define  the  set  of  societal  ideal-types  . . .  admixtures  whose 
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a  priori  probabilities  of  emergence,  persistence  or  reemergence 
are  significant  relative  to  the  universe  of  possibilities 

The  dynamic  focus  of  our  inquiry  concerns  an  analysis  of  the 
conditions  under  which  one  or  another  of  the  societal  ideal- 
types  is  expected  to  dominate  over  the  others.  Each  of  the 
ideal-types  --  in  the  syncretic  framework  —  has  effectively 
unique  ecological  implications  as  well,  these  being  the  nature 
of  its  relationships  with  the  environment  in  which  it  is 
resident..  It  is  these  unique  endogenous  and  environmental 
implications  that  excite  both  the  rise  and  demise  of  the  various 
ideal-types.  The  problem  is  to  deal  with  the  dynamics  of 
societal  configurations  in  a  relatively  disciplined  way,  in 
contrast  to  the  generally  optative  or  normative  predilections 
of  mainstream  macrosociology..  The  key  here  is  the  apodictical 
demand  of  syncretic  methodology,  the  requirement  that  theore¬ 
tical,  logical  or  a  priori  constructs  eventually  submit  to  the 
empirical  discipline  of  normal  science. 

This  study  attempts  to  respond  to  this  dictate  by  summarizing 
its  essentially  qualitative  arguments  and  recasting  them  into 
the  framework  of  a  Markov  model.  This  reformulation  takes  us 
from  the  static-comparative  domain  (where  we  are  concerned 
merely  with  directions  of  influence)  into  a  context  where  we 
must  attempt  to  add  a  quantiative  or  magnitudinal  dimension  to 
the  study.  Particularly  the  Markov  models  direct  our  atten¬ 
tion  to  the  computable  probabilities  of  transition  between  the 
several  societal  ideal-types.  These  transitional  probabilities 
are  initially  established  as  a  priori  (or  judgemental)  indices, 
these  derived  directly  from  the  theoretical  content  of  the  study 
to  this  point.  However,  cnce  such  indices  are  established,  there 
are  certain  statistical  instruments  (notably  variations  on  the 
Bayesian  theme)  that  can  force  the  translation  from  a  priori 
into  objective  (i.e.,  a  posteriori)  transitional  probabilities.; 
So,  in  principle  at  least,  we  have  met  the  syncretic  method¬ 
ology's  demand  for  complementation  between  the  two  generic  arms 
of  sci;ntific  inquiry  deductive  and  inductive  inference. 

To  the  extent  that  the  model's  properties  emerge  satisfactorily 
from  tiie  empirical  validation  process,  the  process  of  macro- 
ciological  change  may  be  brought  a  discipline  and  precision  that 
has  largely  lacked  heretofore.  Major  configurational  changes, 
such  as  those  recently  affecting  Iran  ....  may  in  some  measure 
become  predictable  ...  and,  in  that  regard,  perhaps  manageable 
to  some  extent.  The  essential  arrogance  of  this  assertion  is 
not  lost  on  this  author;  but  the  contention  that  societal 


*!¥■!  .  'U  ,m-~ 


■*?*•?*.  "MM* 


152 

IWAMS 


substance  must  remain  forever  outside  the  bounds  of  scientific 
rationality  is  no  less  arrogant.. 

And  there  is  a  palpable  defense  for  (such  an  approach)..  Its 
essentially  syncretic  quality  means  that  our  models  and  proposi¬ 
tions  remain  as  heuristics  only,  as  constructs  whose  significance 
is  more  methodological  than  substantive. 


Table  10  shows  a  range  of  societal  types  and  their  respec¬ 
tive  units  of  social  significance,  economic  mechanisms  and  political 
structures  as  outlined  by  Sutherland.  Figure  19  is  a  proposed 
normative  model  of  societal  dynamics  developed  by  Sutherland.  The 
Sutherland  concept  clearly  might  prove  useful  in  some  dimensions  of 
the  projection  problem.  In  terms  of  rationale  recording  we  may 
observe  that  cause  and  effect,  similarities  in  systems,  goal-seeking 
and  especially  internal  self-consistency  are  all  implicit  in  the 
rationale  which  underlies  Sutherland's  approach. 


Various  other  structures  of  rationale  are  applicable.  One 
further  example  concerns  rules  for  making  probability  judgments  in 
warning.  Thomas  Belden  has  described  some  rules  as  follows: 


a-  The  more  precise  the  prediction,  the  lower  the 
probability.  If  one  says  A  will  attack  B  on  16  September,  that 
statement  will  have  a  lower  probability  than,  A  will  attack  B 
in  September. 

b.  The  greater  the  number  of  information  elements 
within  the  probability  statement,  the  lower  the  probability. 

For  example,  three  of  A's  divisions  at  X  will  attack  two  of  B's 
divisic"s  at  Y  on  16  September  will  have  a  lower  probability  of 
being  correct  than  A  will  attack  B  on  16  September. 

c.  The  overall  probability  of  the  statement  cannot 
be  greater  than  the  probability  of  any  one  element.  Using  the 
first  example,  in  b.  above,  if  there  is  only  a  30  percent 
probability  that  B  has  two  divisions  at  Y,  then  the  overall 
probability  cannot  exceed  30  percent. 
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Table  10.  Cross-Dimensional  Clustering  of  Social,  Economic 
and  Political  Attributes 


Ideal  Type 

Unit  of  Social 
Significance 

Economic 

(Allocative) 

Mechanism 

Political 

Structure 

1.  Mechanistic 

Tribe 

Segmented  (e.g.,  an 
equal  share  to  each 
head  of  household/ 
proto- Communism 

Oracular 

2.  Proscriptive 

Congregation/ 

parish 

Ascriptive  (e.g.,  as 
with  the  doctrine  of 
the  schoolmen) . 

Oligarchic 

3.  Prescriptive 

Commune/ 

extended 

family 

Requisitional  (to 
each  according  to  his 
need,  etc.)/proto- 
Socialism 

Participatory 

democracy 

4.  Rationalistic 

Nuclear 

family/ 

corporate 

interests 

Algorithmic  (e.g., 
via  supply  and  demand 
or  central  planning) 

Bureacratic  (e.g., 

representative 

democracy) 

5.  Instrumental 

Ad-hocracy/ 

bloc 

Monopolistic/ 

preemptive 

Autocratic 

6 .  Sapient 

Association/ 

conference 

Compensatory 

Collegial 

T 


7.  Existentialists  Isolated  Ascetic  (e.g.,  proto-  Anarchic 

|  individuel  Bhuddist) 
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d.  In  general,  the  greater  the  time  span  of 
prediction,  the  lower  the  probability  of  its  occurrence.  This 
is  so  because  many  events  can  intervene  to  change  the  situation. 

This  requires  that  all  probability  statements  have  the  date  the 
prediction  was  made.. 

Analysis  Routine.  We  should  now  consider  how  the  methods 
and  techniques  of  making  warning  projections  correspond  to  the  steps 
and  procedures  in  the  model  of  warning  analysis  described  in  Section  2 
(see  Table  3,  page  26).  Upon  review  it  is  evident  that  use  of  the 
forms  and  associated  procedures  for  warning  projections  will  lead  the 
analyst  through  the  full  steps,  procedures  and  subprocedures  identi¬ 
fied  in  the  model.  By  focusing  on  the  basic  information  categories, 
the  analyst  can  translate  threat  models  from  the  preceding  stage  into 
projection  formats.  Using  the  Specificity  Rating  Scale,  the  analyst 
can  develop  and  compare  the  specificities  among  various  threat 
estimates.  There  are  also  approaches  by  which  the  analyst  can  develop 
rationale  for  projections.  In  short,  sector  comparison,  probable 
futures  mapping,  divergence  mapping  and  the  other  approaches  represent 
an  integrated  analytic  routine  which  assures  that  the  analyst  will 
think  very  systematically  and  with  considerable  mental  energies  about 
the  prospects  for  various  threats.  This  routine  constitutes  a  defense 
against  the  epistemological  and  cognitive  problems  in  warning  analysis. 
The  backlog  and  relational ity  measures  are  used  to  monitor  the  degree 
to  which  the  analyst  pursues  the  methods  and  techniques  and  follows 
the  steps  and  procedures  in  the  projection  stage. 


More  generally,  we  may  also  observe  that  the  system  of 
analysis  from  monitoring  to  projection  is  a  coherent  analytic  routine 
which  culminates  in  a  set  of  judgments  concerning  which  threats  are 
most  likely  and  Least  likely  at  any  given  time.  We  can  think  of  the 
analyst  as  always  confronting  a  bounded  set  of  possibilities  and 
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attempting  always  to  have  identified  the  more  probable  of  these  and 
related  them  to  threats. 

Signification.  In  discussing  the  threat  recognition  stage, 
it  was  pointed  out  that  signification  is  indicated  when  there  are 
changes  to  and  in  the  threat  models,  for  example,  the  association  of 
new  input  data  with  nodes  in  the  models  such  as  the  critical  event 
filters.  It  was  also  indicated  that  the  full  meaning  assigned  to 
input  data  is  bound  up  in  the  assumptions  underlying  the  models.  A 
distinction  was  drawn  between  high  and  low  context  analysis  and 
their  interactive  relationship  in  the  process  of  warning  analysis. 

It  was  also  seen  that  in  the  projection  stage,  changes  in  meaning 
that  indicate  signification  can  be  recorded  at  a  number  of  points, 
from  the  specificity  ratings  through  the  sector  comparisons,  probable 
futures  maps,  divergence  maps,  decision  stairways,  rationale  matrices, 
rationale  vectors,  and  so  on.  Both  changes  to  and  in  these  analytic 
structures  may  be  recorded.  Further,  to  the  extent  that  the  assump¬ 
tions  underlying  the  threat  modeling  in  the  threat  recognition  stage 
are  developed  in  the  projection  stage,  signification  involving 
development  of,  and  changes  to,  those  assumptions  cross  relate  to 
the  earlier  stage.  In  short,  there  is  obviously  a  considerable 
structure  of  signification  inherent  within  the  full  techniques  and 
methods  of  analysis  from  monitoring  to  projection. 

The  techniques  and  methods  in  all  three  analysis  stages, 
but  especially  in  the  projection  stage,  culminate  in  graphic  and 
alphanumeric  formats  which  represent  a  bort  of  scoreboard  for 
signification.  The  outcome  is  a  series  of  machine-driven  display 
formats  used  to  present  recorded  signification  values  at  various 
levels  and  to  varying  depth.  The  potential  is  there  for  one  to 
examine  the  signification  values  in  a  small  portion  of  a  large 
analysis  or  to  display  in  sequence  a  detailed  and  extensive  summary 
of  a  large-scale  warning  analysis  developed  over  a  significant  period 
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of  time.  Here  the  objective  might  be  to  trace  the  evolution  of 
warning  perspectives,  showing  the  impact  of  input  data  on  the 
rationale  and  assumptions  of  the  analysts;  and  vice  versa,  analysis 
being  an  interactive  tug  of  war  in  which  the  analyst  struggles  to 
fit  the  data  to  his  analytic  structure  with  mixed  success,  the 
failures  causing  him  to  review  and  possibly  recast  portions  of  the 
analytic  structure.  As  discussed  above,  such  a  capability  gives 
rise  to  many  measures  across  a  variety  of  variables.* 

Machine  Support  Implications.  In  the  earlier  remarks  on 
machine  support  implications,  we  discussed  the  general  need  for 
requisite  speed  of  operation,  deferring  detailed  design  requirements 
to  Section  5.  In  the  present  discussion  another  related  and 
extremely  important  requirement  becomes  apparent:  displays  and 
display  techniques. 

Warning  analysis  must  include  a  substantial  visual  dimen¬ 
sion  beyond  oral  and  written  traditions  and  records.  This  visual 
counterpart  must  be  largely  electronic  in  nature.  That  we  need  to 
operate  substantially  in  a  visual  display  mode  in  warning  analysis 
arises  from  several  considerations  we  have  argued  above  and  which 
may  be  summarized  as  the  need  for  speed  of  operation  both  to  aid 
analysis  and  to  aid  measurement  of  analysis. 

Implicit  is  the  theme  that  computer/display  technology 
should  be  harnessed  to  allow  analysts  to  operate  with  extrasomatic 
support  systems  designed  to  complement  both  the  cognitive  processes 


*There  remains  a  highly  technical  (and  often  confusing)  argument  among  theoreticians 
of  interpretation  (or  "Hermeneutics")  over  interpretive  limits,  there  being  some 
schools  such  as  the  Nietzschean  and,  later,  the  Structuralist,  which  tend  to  deny 
that  in  certain  senses  we  can  recover  original  meanings  of  past  records.  See  the 
Notes  section  for  a  brief  discussion. 
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of  analysis  and  the  measurement  of  analysis.  Analysts  must  be 
allowed  to  create  extrasomatic  memories  in  the  form  of  certain  models 
or  schemata,  and  to  manipulate  them  readily  through  interaction  with 
the  machine,  for  purposes  of  changing  the  structures,  recording 
analysis  processes  and  results  for  measurement,  and  recalling  the 
analytic  memory.  Consider  the  fact  that  the  memory  structure  we  have 
just  described  —  PAMNACS,  DENS,  PFMS,  rationale  matrices,  etc.  — 
is  coherent,  considerable  in  scope,  thorough  and,  most  importantly 
for  the  present  thrust,  not  readily  recalled,  let  alone  manipulated, 
by  the  unaided  human  mind:  the  machine  support  is  absolutely  essen¬ 
tial.  Yet  with  the  machine,  and  with  quite  rudimentary  training, 
the  structure  becomes  readily  usable  in  all  dimensions  —  analysis, 
measurement,  and  management.  It  is  in  considering  such  aspects  as 
these  that  the  basic  concept  of  IWAMS  becomes  clearer. 

One  of  the  obviously  essential  applications  in  displays 
is  the  use  of  coLor,  animation,  and  other  cues  to  highlight  changes, 
lend  emphasis  to  portions  of  analysis,  show  threads  of  continuity 
in  analytic  evolutions,  indicate  varying  levels  of  specificity, 
assist  in  memory  recall  operations,  single  out  unresolved  input  data, 
and  many  other  tasks.  There  is  every  prospect  that  color,  animation, 
flashing  cues,  and  other  devices  that  lend  themselves  to  electronic 
displays  will  form  part  of  the  rhetoric,  the  style,  of  the  electronic 
visual  tradition.  As  we  review  the  analysis  output  formats  of  models 
and  maps  of  cognition  in  the  present  approach,  and  imagine  future 
refinements,  it  becomes  clear  that  such  visual  devices  could  become 
extremely  valuable. 


=“53 jg^S&gffiZ'i 


159 

WARNING  ANALYSIS  METHODS 


IWAMS  Informal  Test 


It  is  now  appropriate  to  describe  an  informal  test  of 
the  IWAMS  analytic  system  conducted  at  an  intelligence  center  by  a 
member  of  the  project  team  while  in  residence  for  several  weeks. 

There  were  various  reasons  for  the  test.  Primarily  it  was 
considered  useful  for  developing  software  design  requirements. 
Although  it  was  informal  the  test  also  broadened  the  project  team’s 
understanding  of  aspects  of  warning  analysis  and  management.  By 
informally  reviewing  basic  features  of  the  IWAMS  design  through 
dialogs  with  analysts  and  managers,  the  benefits  of  experienced 
operational  judgments  and  recommendations  were  realized.  Moreover, 
if  only  on  a  manual  basis  the  full  IWAMS  methodological  procedures 
were  tested. 

The  basic  test  procedure  was  as  follows:  First,  typical 
backlogs  of  representative  information  that  might  confront  analysts 
over  given  periods  were  reviewed  and  several  were  assembled. 

The  second  step  was  to  construct  a  mock-up  paper  version 
of  the  IWAMS  analytic  system.  The  types  of  analytic  forms  described 
abwve  were  developed.  The  North  Korean  problem  was  selected  for 
focus  (hence  the  specific  diagrams  and  models  shown  above  apply). 

The  third  step  was  to  conduct  an  informal  analysis  by 
analyzing  the  backlog  data  through  the  routine  of  the  IWAMS  analytic 
system.  Part  of  the  intent  was  to  simulate  a  process  that  would  be 
conducted  with  a  computer-based  system.  For  obvious  reasons,  of 
course,  there  was  no  anticipation  of  total  realism.  However,  records 
analogous  to  log-in/log-out  records  on  a  compurer-based  system  were 
kept.  The  time  used  in  analyzing  input  data  with  various  techniques 
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at  the  different  analytic  stages  was  manually  recorded  as  faithfully 
as  possible.  The  time  spent  away  from  analysis  —  interruptions  that 
might  normally  occur  as  one  worked  such  problems  at  an  analytic 
position  —  were  recorded.  In  addition,  records  were  kept  of  backlog 
at  various  stages;  relationality  as  the  data  moved  through  the  system 
of  analytic  techniques  and  methods;  and  signification.  Examples  are 
provided  below. 

Summary  of  Results.  Since  the  informal  test  was  designed 
to  review  the  IWAMS  concept  in  a  realistic  setting,  the  summary  of 
results  will  concentrate  on  three  widely  different  and  representative 
cases.  The  cases  are  considered  generic  and  have  been  developed  on 
the  basis  of  the  extended  testing.  The  three  cases  are,  first,  a 
description  of  the  detailed  analysis  of  a  typical  single  input; 
second,  a  general  account  of  the  analysis  of  a  substantive  estima¬ 
tive  report  and  the  changes  it  introduced  to  elements  in  the  IWAMS 
analytic  forms  (i.e.,  signification  reflected  in  changes  to  the 
forms);  and,  third,  a  brief  description  of  potential  uses  of  IWAMS 
in  crisis  environments. 

Certain  constraints  should  be  pointed  out: 

•  The  results  of  the  informal  cest  are  qualified  by 
the  fact  that  the  project  team  member,  although  familiar  with  warning 
analytic  operations,  is  not  a  professional  warning  analyst  and  was 
learning  his  own  system  of  analysis  for  the  first  time  in  the  sense 
of  applying  it.  It  must  be  presumed  that  anyone  first  employing  an 
approach,  especially  one  that  involves  a  number  of  steps  and  proce¬ 
dures,  will  become  speedier  at  implementation  with  increased 
experience.  As  we  have  stressed,  use  of  a  computer-based  system 
would  increase  further  the  potential  for  speed.  Therefore,  one 
cannot  assume  that  the  time  frames  involved  in  the  test  in  performing 
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various  analytic  functions,  even  though  relatively  short,  are  neces¬ 
sarily  typical. 

•  Another  important  factor  is  that  during  the  informal 
test  probably  more  changes  were  made  to  the  structure  of  various  forms 
than  might  occur  with  a  system  that  had  been  operational  for  some  time. 
Intuitively  it  seems  probable  that  in  the  latter  case  a  set  of  models 
and  maps  would  be  developed  which  eventually  would  tend  to  staoilize 
(though,  of  course,  never  completely! )  and  ordinarily  be  changed 
steadily  but  minimally,  once  the  initial  hard  work  of  structurir  ; 
them  on  the  basis  of  substantive  analysis  had  been  done.  (Certainly 
this  represents  an  interesting  issue  for  examination. ) 

•  Another  qualification  is  that  the  informal  test  was 
conducted  by  one  parson  over  one  shift  per  day  for  several  days. 
Assuming  a  fully  operational  IWAMS  system,  several  analysts  might  be 
using  it  within  given  shifts,  further,  the  operation  of  IWAMS  might 
develop  into  a  relatively  specialized  job.  For  example,  within  a 
sizeable  cadre  of  analysts  —  say,  a  group  responsible  for  monitoring 
North  Korea  and  Japan  —  a  subset  of  analysts  might  be  responsible 
for  working  against  backlog  and  for  conducting  initial  relationality 
and  signification.  Subsequently  other  analysts  in  the  group  might 
impact  on  the  results  but  not  themselves  follow  the  full  analysis 
paths  through  the  IWAMS  system.  Thus  the  configurations  of  usage 
might  differ  considerably. 

•  In  short,  it  is  not  to  be  inferred  that  the  informal 
test  should  completely  define  "steady  state"  conditions  within  a 
computer-based  IWAMS  system. 
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Case  1 


We  will  begin  by  exemplifying  how  a  single  input  was  moved 
through  the  IWAMS  system  and  the  signification,  relationality,  and 
backlog  measures  obtained.  Section  5  contains  a  description  of  the 
IWAMS  software  design  features,  including  the  specific  man-machine 
interactions  required  in  moving  data  through  the  system.  The  hypo¬ 
thetical  profile  of  analysis  discussed  in  the  present  section  is  a 
functional  description  only.  Implicit  in  many  of  the  steps,  however, 
are  technical  design  features  of  a  computer-based  system. 

The  example  selected  is  this:  assume  receipt  of  a  message 
concerning  the  unusual  deployment  of  a  single  tactical  air  unit  to  a 
specific  airfield.  Assume  that  this  type  of  activity  is  included  in 
the  predefined  list  of  indicators  to  be  watched  in  the  monitoring 
stage . 


Table  11  shows  the  results  of  the  analysis  of  the  message. 

In  reviewing  details  in  the  table,  the  reader  should  recall  the 
severe  expository  problem  of  attempting  to  convey  through  prose  and 
static  diagrams  a  sense  of  speedy  analysis  by  an  experienced  analyst, 
taking  cognizance  of  the  analyst's  background  knowledge  and  the  IWAMS 
high  context  system,  and  assuming  interaction  with  computer-based 
support  whose  operation  the  analyst  has  mastered. 

The  following  discussion  applies  to  the  table.  Note  first 
that  34  steps  or  functions  were  undertaken,  some  of  which  would  entail 
interactions  between  the  analyst  and  the  machine  component  of  IWAMS. 
The  sustained  analysis  time  was  approximately  21  minutes.  Some 
comments  are  useful  here.  First,  as  indicated  above,  this  time  may  be 
lengthier  then  would  be  expected  ordinarily  had  the  analyst  been  using 
an  operational  IWAMS  for  some  time.  Second,  let  us  assume  that  this 
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Table  11.  Analysis  Profile 


*The  "steps,"  "procedures,"  and  "processes"  refer  to  those  in  the  Model  of  Warning 
Analysis  and  comprise  the  analytic  routine. 

**P  =  positive 

N  =  negative 

PD  =  analyst  decision  not  to  pursue  further  analytic  steps. 
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Table  11.  —  Continued. 


Stage 

- 

Analyst  Action 

Time 

Signification 

Relationality 

Backlog 

10..  Check  data  against 

0720 

Step  1, 

PAMNACS  decision 

Procedure  2, 

CEF 

Processes  A-l 

(P) 

11.  Determine  no 

0720 

Signification  (N) 

II 

relevance 

T> 

12.  Tag  data  for  future 

0720 

II 

R 

reference 

0 

T 

G 

H 

13.:  Call-up  DEN  1 

0721 

II 

R 

R 

jr 

E 

14.  Determine  no 

0722 

Signification  (N) 

II 

S 

A 

relevance 

S 

T 

I 

15,  Call-up  DEN  2 

0722 

II 

V 

R 

E 

E 

15.  Determine  no 

0723 

Signification  (N) 

II 

C 

relevance 

R 

0 

E 

G 

17.  Call-up  DEN  3 

0724 

II 

D 

N 

U 

I 

18,  Determine  associa- 

0726 

Signification  (P) 

II 

C 

T 

tion  with  nodes  26 

T 

I 

and  31.  Give 

I 

0 

rationale 

O 

N 

N 

19,  Ask  for  readout 

0729 

II 

other  correlations 

in  DEN  3  with  other 

nodes  over  past 

three  days 

20,  Call-up  DEN  4 

0728 

II 

21,  Determine  no 

0728 

Signification  (N) 

It 

relevance 
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Table  11.  —  Continued. 


Stage 

Analyst  Action 

Time 

Signification 

Relationality 

Backlog  j 

22.  Datermine  none  exist 

0730 

Signification  (N) 

Step  1, 
Procedure  2, 
Processes  A-I 

(P) 

P 

R 

T 

23.  Review  inactive 

0730 

f! 

0 

H 

nodes 

G 

R 

R 

G 

24.  Identify  further 

0731 

II 

E 

A 

information  needs 

S 

T 

S 

25.  Ask  for  other  nodes 

0732 

II 

I 

R 

to  which  data  has 

V 

E 

correlated 

E 

L 

0 

26,  Determine  none 

0732 

Signification  (N) 

1.' 

R 

G 

E 

N 

27,.  Call-up  DEN  3  to 

0732 

II 

D 

I 

which  indicator 

U 

0 

correlates 

C 

N 

T 

28.  Explore  lor  possible 

0734 

Signification  (N) 

II 

I 

alternative  explana- 

0 

tions.:  No  results. 

N 

29,  Decide  net  to 

0734 

Signification  (N) 

Step  2  (PD) 

conduct  novel  threat 

analysis.  Move 
model  forward. 

P 

30,  Call-up  unimpeded 

0735 

Step  1 

P 

R 

prediction  model  for 

Procedure  1 

R  R 

0 

DEN  3 

(P) 

0  E 

J 

G  D 

E 

31.  Determine  no  change 

0735 

Signification  (N) 

Step  1, 

R  U 

C 

in  specificities 

Procedure  2 

E  C 

T 

(P) 

S  T 

I 

S  I 

G 

32,  Indicate  rationale 

0736 

Signification  (N) 

Step  1, 

I  V 

N 

Procedure  3 

V  E 

(P) 

E 

n  m  u  o  so 
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Table  11.  ~  Continued. 


Stage 

Analyst  Action 

Time 

Signification 

Relationality 

Backlog 

P 

33.  Estimate  no  changes 

0736 

Signification  (N) 

Step  1, 

P 

R 

in  remaining  predic- 

Procedures 

R  R 

0 

tion  forms 

4-8  (PD) 

0  E 

J 

G  D 

E 

34.  Decide  no 

0736 

Signification  (N) 

Steps  2,  3, 

R  U 

C 

requirement  pursue 

4  (PD) 

E  C 

T 

additional  projec- 

S  T 

I 

tion  procedures 

S  1 

0 

I  0 

N 

21  minutes 

_ 1 _ 1 _ , 

V  N 

E 

167 

WARNING  ANALYSIS  METHODS 


particular  report  represents  activity  of  an  indicator  of  somewhrt 
more  than  ordinary  significance;  the  analyst  would  likely  treat  the 
indicator  with  greater  than  casual  attention.  Third,  the  example 
chosen  is  appropriate  for  illustrative  purposes  because  it  involves 
input  data  which  an  analyst  would  be  likely  to  pursue  through  a 
number  of  steps  in  the  IWAMS  analytic  procedures.  In  IWAMS  an  analyst 
pursues  data  through  the  system  as  far  as  he  decides  it  is  appropriate 
to  do  so.  There  are  decision  points  at  which  the  analyst  may  decide 
not  to  pursue  the  data  further.  The  data  is  removed  from  the  system 
and  measures  cease.  Of  course  these  cases  are  recorded  within  the 
IWAMS  memory.  (As  seen  in  Table  11  such  cases  are  tagged  in  order 
both  to  be  recalled  if  other  indicators  arise  and  also  tc  be  readily 
identified  in  post  mortems . )  In  our  example,  in  fact,  the  analyst 
does  not  pursue  the  input  data  through  all  the  possible  steps. 

Note  that  in  Table  11  the  points  of  development  of  signi¬ 
fication,  relationality,  and  backlog  measures  are  shown  in  parallel 
with  the  analytic  steps.  There  were  a  total  of  six  steps  in  the 
monitoring  stage,  23  steps  in  the  threat  recognition  stage,  and 
five  steps  in  the  projection  stage. 

Signification.  Several  instances  of  signification  occur 
in  each  analysis  stage.  The  letter,  P,  indicates  that  the  signifi¬ 
cation  was  positive,  meaning  that  the  information  was  correlated 
positively  by  the  analyst,  to  particular  indicators,  model (s),  or 
projection  maps.  The  letter,  N,  refers  to  negative  signification, 
indicating  cases  where  the  analyst  has  decided  that  the  information 
does  not  signify  certain  indicators.  It  is  fundamental  to  determine 
both  types,  there  really  being  no  such  case  as  neutral  signification. 
Either  the  information  has  some  degree  of  positive  meaning  within  the 
context  of  the  indicators,  threat  models  and  projection  maps.,  or  if 
does  not.  In  the  monitoring  stage,  where  tnere  are  four  instances  of 
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signification,  the  first  example  finds  that  the  analyst  predictably 
has  decided  that  from  the  inventory  of  preestablished  indicators  the 
message  specifically  relates  to  indicators  of  air  and  air  defense 
activity.  By  calling  up  only  those  indicators,  signification  has 
occurred:  the  analyst  has  judged  that  if  there  is  a  positive  corre¬ 
lation,  it  must  be  with  those  indicators.  By  not  calling  up  other 
indicators,  which  cover  a  variety  of  other  categories  such  as 
economics,  politics,  and  other  military  indicators,  the  analyst  has 
also  evidenced  negative  signification. 

As  we  move  down  the  column  of  signification  entries  under 
the  monitoring  stage,  we  find  that  the  next  instance  of  signification 
is  positive.  The  analyst  now  determines  that  the  input  data  applies 
to  a  specific  indicator  (called  "C-4"). 

The  next  instance  of  signification  occurs  at  the  fifth  and 
final  step  in  the  monitoring  stage.  The  analyst  now  assigns  a  certain 
activity  level  to  the  indicator  out  of  various  predetermined  levels. 
Let  us  suppose  that  the  analyst  assigns  a  level  of  +1  on  a  scale  of 
1  to  3, 


Notice  that  the  four  instances  of  signification  comprise 
a  succinct,  selective,  chronological  expression  of  the  analytic 
decisions  made  in  the  monitoring  stage  by  our  hypothetical  analyst. 


We  may  now  review  signification  in  the  threat  recognition 
stage.;  All  but.  one  of  the  signification  events  are  negative.  This 
is  not  surprising  since  the  analyst  is  concerned  with  a  single 
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indicator,  reviewing  that  indicator  against  several  diverse  threat 
models.  As  shown  in  Table  11,  the  analyst  assigns  positive  signifi¬ 
cation  at  step  18  and  with  reference  to  DEN  number  3.  Within  that 
particular  model  of  a  possible  North  Korean  threat,  nodes  26  and  31 
presuppose  the  kind  of  activity  that  the  message  indicates  is 
occurring  in  North  Korea.  Until  the  calling  up  of  that  particular 
threat  model,  the  threat  recognition  procedures  had  not  led  the 
analyst  to  determine  any  positive  signification. 

At  this  point,  we  should  review  some  aspects  of  the  threat 
recognition  stage.  Recall  that  the  analyst  departed  the  monitoring 
stage  with  one  indicator  judged  to  be  at  a  certain  level  of  activity. 
The  analyst  then  moved  to  the  threat  recognition  stage  and  called  up 
models  of  projected  alternative  major  national  courses  of  action 
(PAMNACS).  The  analyst  then  checked  the  indicator  data  against  the 
PAMNACS  critical  event  filter  (CEF)  for  policy  review  and  judged  no 
relevance;  he  also  checked  it  against  the  PAMNACS  decision  CEF  and 
judged  no  relevance.  We  should  suppose  that  if  the  analyst  had  been 
on  the  job  for  awhile  and  familiar  with  the  North  Korean  problem,  he 
might  have  gone  straight  to  tne  particular  model  of  relevance  (called 
up  in  step  17).  But  two  points  need  to  be  made.  First,  for  purposes 
of  exemplification  it  was  thought  appropriate  to  depict  the  analyst 
reviewing  the  irdicator  against  several  threat  models.  Second,  there 
should  always  be  a  concern  about  too  much  short  cutting,  based  on 
"familiarity"  with  the  system  of  models.  (The  relationality  measure 
is  designed  in  part  to  monitor  this  practice.) 

Beyond  step  18,  there  is  no  futher  positive  signification 
in  the  threat  modeling  stage.  In  our  hypothetical  analysis  situation, 
this  fact  would  suggest  that  in  the  absence  of  other  threatening  indi¬ 
cators  the  unusual  deployment  may  be  an  isolated  event  rather  than 
signifying  the  emergence  of  a  threatening  situation  in  North  Korea. 
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In  step  21,  the  analyst  asks  the  system  to  tell  him  whether  in  the 
past  three  days  there  have  been  other  such  indicators,  perhaps  in  the 
ground  forces,  correlating  to  other  nodes  within  the  particular  DEN. 
Seeing  none  of  them  (step  22),  the  analyst  then  asks  for  display  of 
the  inactive  nodes  in  the  DEN  in  question  and  sees  that  they  are 
numerous.  In  step  25,  he  then  asks  for  a  call-up  of  other  models  to 
which  the  data  has  correlated  and  sees  that  there  have  been  none. 
Therefore,  by  the  time  he  reaches  step  29,  the  analyst  has  decided 
not  to  conduct  a  novel  threat  analysis  but  to  move  the  DEN  in  question 
forward  to  the  projection  stage.  The  analyst  still  has  the  obligation 
to  determine  whether  the  activity  associated  with  DEN  number  3  would 
signify  any  change  in  the  status  of  projections. 


In  the  projection  stage,  the  analyst  undertakes  five 
analysis  steps.  First  he  calls  up  the  prediction  format  for  DEN  3. 
(Recall  that  the  DENs  are  translated  into  a  predictive  format 
organized  around  the  basic  information  categories.)  In  step  31 
the  analyst  determines  no  change  in  specificities  across  the 
categories,  who,  what,  when,  where,  how,  and  why.  As  is  evident  in 
the  remainder  of  the  projection  stage,  the  analyst  then  decides  not 
to  pursue  the  analysis  further.  He  therefore  has  made  the  decision 
not  to  move  from  the  unimpeded  projection  to  the  influenced  projection 
procedures,  nor  from  there  to  the  unimpeded  and  influenced  forecast 
stages. 


The  analyst  has  judged  that  the  situation  is  essentially 
"normal"  in  North  Korea,  in  spite  of  the  fact  that  there  has  been  an 
unusual  deployment  of  a  single  tactical  air  unit.  For  the  present, 
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the  significance  of  the  input  —  the  meaning  associated  with  it  —  is 
that  it  does  not  fundamentally  alter  any  of  the  current  projections 
and  probabilities.  The  North  Koreans  appear  still  in  basically  the 
same  posture  they  were  in  before  news  of  the  unusual  deployment.  Yet 
the  analyst  has  been  alerted  because  the  indicator  is  deemed  poten¬ 
tially  significant.  Obviously  there  are  variations  to  this  basic 
case  in  which  the  situation  might  have  been  viewed  as  considerably 
different.  In  a  developing  crisis  situation,  for  example,  analytic 
reactions  and  responses  would  have  been  different,  particularly  in 
view  of  the  fact  that  a  number  of  other  threatening  indicators  in 
political,  military,  and  other  dimensions  might  well  have  been 
developing. 

Relationality.  The  relational ity  measurement  is  made  to 
determine  how  fully  the  routine  —  the  various  steps  and  procedures 
of  analysis  —  is  followed  by  the  analyst.  The  key  means  of  moni¬ 
toring  are  the  recorded  instances  of  man-machine  interactions  between 
the  analyst  and  the  IWAMS  system  as  these  interactions  correspond  to 
and  reflect  analysis  procedures  and  steps.  Hence  the  data  needed  to 
measure  relationality  develops  as  the  analyst  performs  certain  func¬ 
tions  using  the  machine,  such  as:  calling  up  lists  of  activity 
indicators;  assigning  levels  of  activity  to  indicators  and  entering 
them  into  the  system;  calling  up  various  threat  models;  changing  the 
configuration  of  the  threat  models;  and  so  on  through  the  full  range 
of  procedures  which  culminate  in  the  projection  stage. 

Under  the  monitoring  stage,  there  are  seven  positive 
instances  of  relationality.  Steps  1  through  7  of  the  analysis  routine 
have  been  followed  by  the  analyst  in  analyzing  the  example  message. 
Steps  8  and  9  were  not  followed  and,  therefore,  show  negative  relation 
ality.  Steps  8  and  9  pertain  to  analysis  of  novel  threat  situations; 
since  the  report  being  examined  did  not  at  present  seem  to  signify  in 
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and  of  itself  a  novel  situation,  it  was  not  so  analyzed.  However,  the 
later  tagging  of  the  input  message  (see  step  12  under  threat  recogni¬ 
tion)  allows  it  to  be  reviewed  should  a  significant  amount  of  other 
information  on  unusual  events  accumulate  over  a  short  time  period. 

The  decision  not  to  follow  novel  threat  analysis  steps  is  also 
recorded  in  the  memory. 


We  may  now  consider  the  threat  recognition  stage.  Rela- 
txonality  is  quite  comprehensive  until  step  29,  where  the  analyst 
ag^in  has  decided  not  to  pursue  a  novel  threat  analysis.  Note  that 
the  reiationality  entry  in  the  table  indicates  that  step  2  has  in 
fact  been  addressed  at  that  point,  but  negatively,  as  symbolized  by 
PD, 


In  the  projection  stage,  steps  33  and  34  again  show  a  PD 
status  with  respect  to  reiationality.  The  analyst  has  decided  that 
since  he  has  judged  negative  signification  to  that  point,  it  is 
unnecessary  to  call  up  and  pursue  further  projection  tecnniques  and 
analytic  steps.  Specifically,  he  has  determined  that  the  input  has 
no  positive  signification  with  respect  to  the  unimpeded  prediction 
models,  and  therefore  that  no  purpose  is  served  by  initiating  influ¬ 
enced  prediction  procedures  or  unimpeded  and  influenced  forecast 
procedures.  This  analytic  decision  was  based  on  signification  and 
noi;.  the  result  of  lack  of  time  or  other  problems  that  could  reduce 
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relationaiity.  The  IWAMS  design  includes  the  capability  to  record 
in  the  memory  whether  relationaiity  was  cut  short  because  of  signi¬ 
fication  in  an  ensuing  analyst  judgment  or  whether  for  some  other 
reason.  Hence  the  PD  designation  is  meant  to  indicate  those  instances 
in  which  the  analyst  makes  a  conscious  decision,  based  on  the  meaning 
given  the  data,  not  to  pursue  relationaiity  further. 

Backlog.  Backlog  values  would  also  be  shown  in  the 
analytic  process.  Backlog  will  simply  be  shifting  and  reducing  as 
a  function  of  examining  inputs  through  relating  them  to  the  analytic 
procedures.  For  example,  at  the  end  of  step  6  in  the  monitoring 
stage  and  at  the  beginning  of  the  threat  recognition  stage,  which 
occurs  in  step  7,  the  backlog  with  respect  to  the  monitoring  stage 
of  analysis  would  be  reduced  by  at  least  one,  that  is,  by  the  example 
report.  Were  an  analyst  or  manager  to  wish  to  examine  backlog  at 
any  point  in  the  process,  the  information  would  be  there  to  calculate 
and  display  backlog  for  individual  steps  and  procedures. 


Uses  of  the  Measures  Data 


Before  proceeding  to  examine  the  other  two  analytic  cases, 
it  is  appropriate  to  discuss  potential  uses  of  the  measurement  data 
retained  in  the  IWAMS  memory.  It  is  useful  to  imagine  that  the 
outputs  from  the  different  stages  of  analysis  are  recorded  in  an 
overall  memory  consisting  essentially  of  three  submemories.  The 
first  submemory  relates  to  information  that  intelligence  managers 
might  have  the  greatest  interest  in.  The  second  submemory  would 
contain  information  on  the  analytic  process  that  might  be  of  primary 
interest  to  the  analysts  themselves.  One  might  designate  this 

»  submemory  as  an  analytic  working  memory.  The  third  submemory  would 
consist  of  the  accumulation  of  data  that  might  be  of  interest  for 

1 
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operations  research  and  R&D  aimed  at  improving  the  conceptual  under¬ 
standing  of,  and  technological  support  to,  warning  operations. 

The  memory  subdivisions  are,  of  course,  arbitrary.  It  is 
certainly  conceivable  that  all  three  users  —  managers,  analysts,  and 
researchers  —  would  want  access  to  portions  of  all  the  submemories. 
Obviously,  however,  a  certain  degree  of  privacy  must  be  maintained, 
so  that  the  memory  would  not  be  totally  open,  say,  to  the  R&D 
community.  However,  some  of  the  data  would  serve  all  three  users. 


At  least  some  ma]or  distinctions  can  now  be  made  as  to  what 
should  be  provided  for  various  users.  For  the  managers  of  I&W  opera¬ 
tions,  for  example,  probably  daily  and  other  cumulative  displays  of 
backlog  should  be  provided.  Managers  should  be  able  to  detect  where 
backlogs  exist,  their  size,  rates  of  reduction  in  the  past  versus 
resources,  and  other  such  data  which,  when  compared  to  priorities, 
would  help  in  the  day-to-day  management  of  operations.  In  general 
(but  not,  of  course,  always)  managers  presumably  would  wish  to  monitor 
operations  on  a  broad  scale:  for  instance,  total  backlog,  backlog  by 
area,  relationality  trends,  and  major  changes  in  signification  (both 
within  and  to  the  structures  and  forms  of  analysis).  Often  managers 
will  be  looking  for  indications  of  notable  changes  from  the  norm. 


lit. 
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When  we  speak  of  "management, "  we  must  stress  that  the 
analyst  is  a  manager  too.  Analysts  will,  in  a  fundamental  sense, 
manage  their  own  analysis.  Analysts  must  be  able  to  monitor  the 
uses  they  have  put  data  to;  the  uses  it  has  been  put  to  by  other 
analysts  within  the  group;  by  analysts  on  shifts  other  than  their 
own;  and,  ultimately,  by  analysts  at  other  nodes  in  the  national 
analytic  network.  Analysts  must  be  able  periodically  to  review  the 
changes  in  and  to  models  and  procedures.  The  IWAMS  system,  therefore, 
is  designed  to  summarize  analytic  operations,  particularly  in  terms 
of  signification,  relationality  ar.d  backlog,  for  working  analysts. 
Potential  forms  of  the  summaries  are  powerful  and  fascinating.  For 
example,  the  signification  changes  over  a  given  period  could  be 
displayed  through  the  models  and  cognitive  maps  already  in  the 
computer.  Color  and  other  display  techniques  such  as  dynamic  mapping 
and  flashing  could  be  used  to  highlight  changes  dramatically  and 
effectively.  Analysts  would  examine  rapidly,  in  telescoped  fashion, 
the  analytic  processes  for  many  ranges  of  time  with  respect  to  various 
problems.  This  would  be  highly  useful  to  analysts  in  such  functions 
as:  monitoring  their  own  operations;  monitoring  the  types  and  impact 
of  data  they  regularly  receive;  and  monitoring  threat  dynamics. 

Concei/ably  both  managers  and  decisionmakers  being  briefed 
might  wish  such  summaries  and  demonstrations  of  signification  and 
relationality.  To  cite  one  of  many  possible  examples,  the  onset  of 
a  threatening  si  ;uation  in  a  given  area  might  lead  managers  to  monitor 
rapidly  the  state  of  analytic  judgment  and  perspective  on  the  threat 
by  utilizing  summaries  based  primarily  on  sequences  of  displays, 
supplemented  by  discussions  with  the  analysts. 


A  third  use  of  the  IWAMS  memory  is  for  purposes  of  off-line 
operations  research  and  the  development  of  R&D  requirements.  We  are 
referring  primarily  to  the  measurement  of  a  number  of  variables 
representing  key  points  in  the  cognitive  process  of  analysis.  They 
concern  aspects  such  as  the  rates  of  flow  of  information  within  the 
system,  the  relationship  between  types  of  data  and  types  of  analytic 
operations,  and  many  other  variables.  Data  on  such  variables  should 
be  available  for  research  off-line  to  provide  insight  into  the  kinds 
of  technology  that  should  be  developed  and  transferred  to  support 
warning  analysis  operations. 


Case  2 


Case  1,  a  very  simple,  straightforward  example  involving 
one  message,  was  chosen  as  an  appropriate  introduction  to  the  opera 
tion  of  IWAMS.  Another  classic  case  is  that  in  which  the  analyst 
must  review  ar.d  respond  to  a  probing,  broad  estimate  of  likely 
developments  in  his  sphere  of  interest.  Intelligence  estimates 
made  at  the  national  level  which  concern  possible  developments  in 
various  world  regions  are  examples  of  such  studies .  These  substan¬ 
tive,  far-reaching  estimates,  made  by  various  agencies  and  offices, 
are  prepared  periodically  and  should  be  utilized  by  the  warning 
analyst  as  a  source  of  the  development  of  the  high  context  analytic 
structure  of  models,  assumptions,  etc.  In  addition,  such  studies 
and  research  conducted  outside  the  government  should  also  be  used. 


During  the  informal  test  of  IWAMS,  several  such  studies  and 
estimates  concerning  Asian  affairs,  notably  centering  on  the  Korean 
peninsula,  were  reviewed.  As  a  means  of  exemplifying  how  they 
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would  be  utilized  with  IWAMS,  we  will  postulate  a  hypothetical  esti¬ 
mate  concern inq  Korean  affairs,  as  follows.  Assume  that  the  analyst 
reviews  a  detailed  analysis  of  the  political,  economic,  military, 
and  other  constraints  on  both  North  and  South  Korea  with  respect  to 
policy  over  the  near  term.  The  estimate  probes  the  interrelationships 
among  economic  problems,  political  problems  (domestic  and  interna¬ 
tional),  military  force  shortfalls,  and  numerous  other  aspects, 
particularly  those  influencing  the  policy  postures  taken  by  the 
leadership  in  North  Korea.  Let  us  also  assume  that  regardless  of 
the  extent  to  which  given  analysts  might  subscribe  to  the  assump¬ 
tions  and  judgments  in  the  report,  it  is  nevertheless  generally  a 
very  comprehensive  and  useful  input. 

In  such  cases  (and  based  on  the  experience  of  the  project 
team  member  conducting  the  informal  test  of  IWAMS)  an  analyst  typi¬ 
cally  might  spend  several  hours  reading  and  analyzing  one  such 
estimate,  breaking  out  what  appear  to  be  the  salient  points  for  the 
warning  analytic  structure  of  models,  sectors,  probability  maps, 
assumptions,  etc.  Based  again  on  the  experience  during  the  test, 
it  appears  that  .if  there  is  a  system  of  analysis,  analysts  ordinarily 
might  deal  with  daily  message  traffic  in  blocks  of  time  during  a 
shift  and  use  other  time  blocks  for  review  of  estimates  (which  appear 
only  periodically)  and  other  such  inputs  and  for  working  on  analytic 
structure,  methods,  and  procedures.  Of  course,  functional  distinc¬ 
tions  between  watch  personnel  and  analysts  performing  detailed  review 
oi  data  are  obviously  germane  here.  So  is  the  question  of  the  role 
of  IWAMS  specialists. 

Regardless,  estimative  reports  will  impact  most  strongly 
on  the  threat  models  and  particularly  on  the  structures  and  assump¬ 
tions  in  the  projection  stage.  For  example,  a  new  sector  might  be 
added  to  the  list  of  sectors  of  North  Korea.  Assume  that  prior  to 
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his  analysis  of  our  hypothetical  estimate,  the  analyst  had  estab¬ 
lished  these  sectors:  E  (economic  status);  B  (perceptions  of  North 
Korean  decision  makers);  and  M  (military  force  status  vis-a-vis 
South  Korea).  Assume  that  based  on  reviewing  the  estimate,  a  new 
sector,  I,  the  degree  of  political  control  exercised  by  the  current 
North  Korean  regime,  is  added  to  the  sectors. 

An  important  point  is  that  the  addition  of  this  new  sector 
creates  ramifications  throughout  the  remaining  structures  of  the 
projection  stage.  For  example,  a  new  single  sector  map  must  now  be 
created  to  accommodate  the  new  sector,  and  this  will  create  a  general 
shift  in  perspectives.  Further,  the  probable  futures  maps,  which 
encompass  all  sectors,  must  be  changed.  In  turn,  the  rationale 
matrices  will  chmge,  as  will  the  assumptions,  and  so  on. 

Ramification  in  Meaning.  We  new  arrive  at,  and  must  digress 
to  discuss,  an  extremely  important  aspect  of  signification  (and  of 
warning  analysis  per  se).  Let  us  call  it  the  Condition  of  Ramified 
Meaning.  What  we  have  just  described  is  a  process  whereby  an 
analyst's  decision  about  signification  • —  about  a  change  in  meaning  — 
occurs  (as  it  must  in  the  sequential  world,-  at  a  single  point  in  the 
analytic  system.  It  is  focused  on  some  element  in  the  high  context 
system.  These  points  now  become  critical: 

1.  Wiether  or  not  it  is  noticed  and  recorded,  the  change 
at  the  single  point  in  the  stricture  will  to  the  degree  that  the 
whole  structure  ls  coherent  result  in  far-reaching  ramifications  in 
meaning.  New  me  ming  will  occur,  independently  of  human  recognition 
of  it  (various  philosophical  viewpoints  notwithstanding).  In  effect, 
then,  we  can  think  of  an  extrasomatic  memory  as  having  a  coherent  set 
of  perspectives,  and  that  this  set  changes  to  some  extent  as  a  result 
of  a  human  analyst  assigning  meaning  somewhere  in  the  set. 
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2.  There  is  a  positive  relationship  between  the  principle 
of  coherence,  as  applied  to  the  system  and  structures  of  analysis, 
and  the  dynamics  of  the  ramification  of  meaning. 

3.  Among  the  specific  cognitive  problems  of  the  unaided 
human  mind  are  its  limitations  in  "seeing"  the  extent  of  ramifica¬ 
tions  in  meaning,  particularly  in  timely  fashion. 


4.  The  real  issue  becomes  that  of  aiding  the  mind  by 
using  a  computer-based  system  programmed  to  (a)  indicate  with  color- 
based  or  other  prompting  cues  when  ramifications  are  probably 
consequent,  so  that  the  analyst  may  realize  it  and  explore  them;  and 
(b)  as  feasible,  contain  algorithms  which  develop,  record,  and 
display  the  ramifications.  Obviously,  in  many  cases  it  will  be  the 
analyst  who,  once  notified,  is  obliged  to  confront  and  resolve  a«  best 
he  can  the  implications  of  individual  acts  of  signification. 


5 .  These  factors  put  into  better  perspective  the  require¬ 
ment  for  computer-based  technological  support  to  the  unaided  human 
mind  attempting  to  perform  complex  and  extremely  difficult  warning 
analysis.  We  now  have  additional  context  for  the  earlier  discussions 
of  signification. 


Returning  to  the  example  in  Case  2,  we  may  consider  a 
procedural  question.  By  what  procedures  and  mechanisms  should 
analysts  and  groups  of  analysts  change  analytic-  structure,  for 
'  example,  change  the  number  of  nodes  and  indicators  in  a  threat 

J  model?  The  authors  have  no  detailed  recommendations,  but  a 

suggestion.  Analysts  at  least,  ought  to  be  able  to  make  such 
changes  informally  and  store  them  in  working  or  activity  files 
provided  by  their  computer-based  s’apport  system.  Eventually, 
through  established  procedures,  decisions  on  formal  changes  could 
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be  made.  Certainly  individual  warning  judgments  based  on  different 
analytic  perspectives  should  be  recorded  and  available  for  review. 
Further,  models  will,  as  we  have  stressed,  tend  to  get  out  of  date. 

It  is  important  to  monitor  the  status  of  the  analytic  structure  with 
respect  to  the  relative  ages  of  various  portions  of  it. 

Various  inputs  to  analysts  can  cause  change  to  the  structure 
of  analysis  models.  Open  literature  in  fields  such  as  political 
science,  economics,  forecasting,  and  dynamic  modeling  may  be  used  by 
analysts  to  change  the  structure  of  analysis  procedures  and  models. 

The  same  is  true  of  consultation  with  experts.  And  we  need  hardly 
mention  the  analyst's  own  experience  on  the  job. 


Case  3 


Another  important  case  is  the  crisis  situation.  In  this 
instance,  a  number  of  indicators  such  as  the  unusual  deployment  of 
tactical  aircraft  would  presumably  be  input  to  the  analyst.  The 
process  describe  I  in  Case  1  would  be  greatly  amplified  and  intensified. 
A  number  of  analysts  would  be  working  the  problem,  inputting  different 
indicators,  for  example,  from  different  desks  and  at  different  times 
of  the  day.  A  major  value  of  IWAMS  would  be  its  ability  to  accumulate 
and  summarize  quickly  the  multiple  inputs  and  analytic  judgments  from 
different  analysts. 


Trar sition 


We  have  now  reviewed  the  structure  and  dynamics  of  the 
IWAMS  analytic  s/stem.  This  system  is  designed  basically  to  provide 
the  analytic  measures  necessary  to  monitor  effectiveness,  readiness 
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and  maintenance  values.,  and  to  foster  effective  analysis.  The  next 
section  will  document  the  software  design  of  the  IWAMS  system.  This 
design  consists  of  depicting  the  file  architecture  and  programming 
reguired  to  bring  IWAMS  to  fruition. 


5 
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The  present  section  describes  a  software  system  for  IWAMS. 
Figure  20  shows  the  major  emphasis  in  the  description  relative  to 
the  total  IWAMS  concept.  Portions  of  the  discussions  in  Sections  2, 
3,  and  4  bear  on  the  functional  design  of  IWAMS.  The  definition  in 
Section  3  of  measures  of  warning  analysis,  and  the  development  in 
Section  4  of  the  analysis  procedures,  forms,  and  graphic  displays  of 
analysis  output,  apply  directly  to  the  present  description.  Especi¬ 
ally  pertinent  is  the  informal  test  of  IWAMS. 

Figure  21  shows  the  overall  functional  design  of  the  IWAMS 
software.  The  present  discussion  is  organized  around  the  figure. 
Design  requirements  for  each  of  the  eight  major  functions  are 
discussed,  together  with  details  on  the  subfunctions.  Writeups  are 
provided  on  the  design  requirements  for  analytical  modeling,  for 
developing  and  displaying  measures,  for  developing  and  modifying 
models  and  projection  maps,  for  providing  management  access  to  the 
status  of  analysis,  for  data  base  retrieval  functions,  for  logging 
operations,  and  for  operations  with  input  data.  In  the  vernacular 
of  MIL  Standard  7935. 1-S  (Automated  Data  Systems  Documentation 
Standard),  the  design  description  represents  a  Functional  Description 
document.  The  design  presented  can  serve  as  a  basis  for  extended 
development  efforts. 

The  IWAMS  design  description  assumes  the  existence  of 
computer-based  analysis  support  systems  at  major  warning  centers; 
IWAMS  would  be  integrated  into  those  systems.  Therefore,  the  design 
does  not  include  hardware  or  computer  architecture  considerations. 

The  remainder  of  this  section  will  consider  in  turn  each 
element  in  Figure  21. 
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Figure  21.  Basic  Design  of  IWAMS 
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Host  System  File  Architecture 


It  may  be  assumed  that  in  warning  centers  the  host  analysis 
support  system  --  that  system  into  which  IWAMS  would  be  integrated  -- 
will  include  two  specific  files  for  use  by  analysts  which  also  could 
be  used  by  selected  managers.  These  files  are  designated  activity 
files  and  work  files.  In  Figure  21  under  the  block  entitled.  Host 
File  Structure,  we  have  listed  work  files,  activity  files,  and  other 
files.  The  third  category  is  included  in  the  event  that  specialized 
files  for  IWAMS  might  be  created,  particularly  for  management. 


With  respect  to  the  format  for  the  two  major  files,  two 
items  are  particularly  noteworthy:  first,  a  substantial  amount  of 
input  data  will  be  entered  into  IWAMS  in  various  forms  (to  be 
discussed  subsequently);  second,  all  analysis  activity  and  direction 
is  captured  either  by  explicit  man-machine  interaction,  such  as 
analysts  logging  on  the  system  and  interacting  with  the  system  in 
various  ways  prior  to  logging  off;  or  by  nonintrusive  programming, 
such  as  recording  the  lengths  of  time  within  th^  computer  over  which 
analysts  perform  such  functions  as  modifying  ^  elements  in  threat 
models  (and  thereby  developing  signification).  With  this  background, 
we  will  now  discuss  the  specific  file  structures. 


Analyst  Activity  Fils.  Two  types  of  subfiles  are  used 
under  this  category.  The  first  will  hold  input  data  time  tagged  and 
listed  by  type.  Input  data  will  be  listed  under  categories  such  as 
message,  document,  daily,  weekly,  monthly,  and/or  by  subject  matter, 
producing  agencies,  classification,  etc.  Further,  the  file  will 
contain  the  text  of  uhe  messages  and  documents  if  it  is  short  enough; 
or  the  reference  number,  with  an  analyst-generated  synopsis.  Often 
the  information  from  a  document  that  might  be  pertinent  will  be  brief; 
it  may  consist  of  an  excerpt  from  a  relatively  substantial  document. 
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The  second  subfile  under  the  activity  file  category  will 
provide  a  list  of  the  monitoring  indicator  structures,  the  various 
threat  models,  and  the  different  projection  maps  and  matrices 
currently  used  by  the  given  analyst(s).  In  addition,  this  subfile 
will  contain  references  to  analytic  structures  being  used  by  other 
analysts .  These  other  analysts  may  be  located  within  the  same 
intelligence  center  or,  depending  on  the  extent  to  which  the  system 
is  deployed,  resident  at  other  geographic  locations. 

Analyst  Work  Files.  Again,  two  subfiles  will  be  provided. 
One  will  contain  the  informal  models  used  by  analysts.  There  is  a 
presumption  here  that  individual  analysts  should  have  the  capability 
to  work  with  their  own  models,  projection  maps,  matrices  and  other 
forms,  and  to  store  and  have  access  to  these  within  their  work  files. 
In  addition,  it  would  probably  be  useful  to  establish  one  set  of 
models  and  other  analytic  structures  which  reflect  the  formal  analytic 
perspective  for  given  groups  of  analysts.  Thus  it  may  be  useful  to 
distinguish  between  informal  and  formal  models,  and  to  include  in  the 
design  the  capability  to  store  and  work  with  both  kinds.  As  suggested 
above,  decisions  on  changing  formal  analytic  perspectives  might  follow 
criteria  developed  by  management. 

The  second  subfile  would  contain  a  record  of  the  analytic 
actions  performed  on  input  data  as  reflected  in  changes  to  and  within 
the  analysis  models,  maps,  matrices,  etc.  This  particular  file  would 
store  the  signification  values,  including  their  rationale,  as 
developed  by  the  analyst.  Implicit  here  is  that  the  analyst  provides 
through  manual  input  some  of  the  rationale  descriptions  needed.  For 
example,  in  Section  4  matrices  are  described  which  serve  as  a  format 
for  expressing  the  rationale  behind  various  analytic  perspectives. 

This  storage  capability  applies  both  in  the  case  of  the  individual 
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analyst  and  in  the  case  of  files  containing  the  formal  perspectives 
of  a  given  analytic  group. 

Management  Activity  File.  This  file  will  contain  time- tagged 
measures  such  as  backlog  and  relationality  which  have  been  generated 
through  analytic  activity.  The  manager  will  have  access  to  a  file 
which  ultimately  provides  to  him  by  appropriate  sequences  of  geometric 
and  alphanumeric  displays  the  data  on  backlog,  relationality,  and 
signification  values,  together  with  translations  of  these  to  values 
in  effectiveness,  readiness  and  maintenance.  This  management  file 
will  include  a  measure  value  action  pointer  which  points  to  records 
appropriate  for  review  within  work  files.  Depending  on  the  design 
judgments  made  by  analytic  management,  access  could  be  provided  to 
both  formal  and  informal  analytic  structures. 

Management  Work  File.  Depending  on  the  desires  of  manage¬ 
ment,  it  would  also  be  feasible  to  structure  a  management  work  file. 

The  purpose  of  this  file  would  be  to  enable  management  to  interconnect 
with  analyst  files  and  convey  priorities  ana  directions  to  given 
analysts.  Whether  this  should  be  done  in  part  through  computer 
operations  or  done  simply  through  verbal  means  is  a  design  issue  to 
be  decided  within  the  operational  context. 


Data  Base  Retrieval 


The  data  base  retrieval  mechanisms  to  be  used  in  IWAMS 
will  be  the  same  as  the  mechanisms  used  in  the  host  system.  Obviously 
it  will  be  necessary  to  review  the  host  system  retrival  mechanism 
prior  to  IWAMS  integration  in  order  to  assure  that  there  are  no 
incompatibilities . 
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Input 


Both  automatic  and  manual  inputs  will  be  provided.  Auto¬ 
matic  input  modes  presuppose  that  portions  of  certain  kinds  of  input, 
such  as  message  traffic,  are  automatically  input  to  the  overall 
analytic  support  system.  The  format  for  both  types  of  input  would 
be  the  same:  header  followed  by  text.  The  record  length  would  be 
variable.  The  header  portions  would  conform  to  those  which  are 
currently  used  with  the  various  traffic  coming  into  the  warning  center. 
The  data  itself  would  be  stored  in  the  computer  as  American  Standards 
Computer  Interchange  (ASCII)  data. 

Manual  Input.  Manual  input  capabilities  will  be  necessary 
to  provide  a  mechanism  with  which  analysts  may  interact  with,  and 
supplement,  automatic  input  data.  More  basically,  analysts  will  need 
the  manual  input  capability  because  they  will  need  to  excerpt  and 
extract  information  from  various  inputs.  Analysts  must  be  able  to 
tailor  the  input  they  wish  to  store  within  the  IWAMS  system  since, 
as  indicated  in  Section  4,  there  are  in  many  cases  special  formats 
for  the  data  entered  into  the  system  of  models,  projection  maps, 
matrices,  etc.  Given  the  volumes  of  input  data,  IWAMS  specialists 
may  provide  support  for  manual  input.  Moreover,  lists  of  documents 
received  by  the  library  facilities  at  intelligence  centers  could  be 
entered  into  the  system  by  library  personnel  and  routed  to  the 
appropriate  analytic  files. 

Automatic  Input.  Various  types  of  input  data  would  be 
automatically  available  to  analysts.  With  respect  to  the  format  for 
automatic  input  data,  some  message  type  traffic  might  include  the 
actual  text.  In  the  case  of  other  kinds  of  input,  a  reference  to 
the  document  number  and  title  might  be  automatically  generated, 
with  the  actual  perusal  of  the  document  made  after  manual  retrieval 
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from  the  holding  office.  The  synopsizecl  results  of  that  review  could 
be  input  to  the  system  manually  and  form  part  of  the  record. 

Ultimately  the  analyst  would  enter  some  data  within  the 
context  of  the  models,  projection  maps,  assumption  matrices,  aiid 
other  forms  which  comprise  the  overall  analytic  structure.  Thus 
automatic  and  manual  inputs  to  the  system  really  occur  at  several 
levels,  beginning  with  initial  input  of  information  for  review, 
followed  by  routing  the  information  (in  whatever  reduced  or  modified 
form)  for  further  input  to  specific  nodes  and  elements  within  the 
formal  structure  of  analysis. 


IWAMS  Analysis  System 

The  IWAMS  analysis  system  was  described  in  detail  in 
Section  4.  The  present  section  will  describe  the  computer  implemen¬ 
tation  of  that  system. 

Monitoring.  As  noted,  the  analyst  may  be  supported  by 
various  computer-based  qualitative  and  quantitative  techniques  for 
monitoring  indicators.  It  may  be  that  the  analyst  is  already  able 
to  display  lists  of  key  indicators  through  the  basic  support  system. 
In  addition,  quantitative  analytic  techniques  which  involve  assigning 
priorities  and  weightings  to  various  indicators  and  combinations  of 
indicators  may  also  be  available  within  the  basic  support  system. 
Other  quantitative  techniques  for  monitoring  indicators  may  also 
become  available.  Whatever  the  status  of  such  support,  the  IWAMS 
system  will  either  have  access  to  existing  techniques  or  provide 
through  its  files  the  capability  to  store  what  is  required.  IWAMS 
will  be  designed  to  assist  the  analyst  in  whatever  functions  are 
entailed,  such  as  (1)  review  incoming  data  applicable  to  his  assigned 
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activity  or  region;  (2)  extract  the  pertinent  information;  (3)  corre¬ 
late  the  information  against  indicator  lists;  and  (4)  generate  a 
result  in  terms  of  graphics  displays  of  active  indicators.  Depending 
on  the  quantitative  techniques  available  to  the  analyst,  the  result 
generation  may  involve  a  statistical  presentation. 

With  respect  to  the  review  of  input,  the  analyst,  having 
stored  data  in  IWAMS,  can  readily  extract  and  display  continually 
updated  lists.  Analysts  can  then  review  these  lists,  extracting 
individual  data  as  necessary. 


Information  extraction  with  IWAMS  would  be  primarily 
manual.  However,  the  system  does  furnish  assistance  insofar  as  it 
provides  current  activities,  indications,  or  data  points  that  must 
be  verified,  checked,  or  otherwise  correlated  against.  This  function 
gives  rise  to  the  important  concept  that  the  IWAMS  system  include 
the  capability  to  develop  prompting  and  cueing  of  the  analyst, 
through  the  use  of  color  and  other  techniques  such  as  flashing  and 
underscoring  on  the  requisite  display  formats. 

The  data  correlation  activity  is  also  primarily  manual. 
Ultimately  the  system  will  assist  analysts  because  it  will  capture 
and  record  the  results  of  previous  correlations.  The  IWAMS  system 
will  also  include  prompting  devices  to  help  the  analyst  determine 
what  steps  and  procedures  in  the  analytic  routine  might  be  followed 
as  a  result  of  correlations  made  to  date. 

Output  activity  would  be  automated  in  IWAMS.  The  outputs 
could  take  the  form  of  various  lists  of  indicators;  and  alphanumeric 
and  statistical  renderings  of  the  past  and  present  status  of  analytic 
operations  in  the  monitoring  stage. 


192 

IWAMS 


Threat  Recognition.  Considerable  detail  was  provided  in 
Section  4  on  the  threat  recognition  stage-.  The  analyst  seeks  to 
account  for  indicator  activity  from  the  monitoring  stage  by  reviewing 
the  activity  against  an  interrelated  set  of  models  (or  information 
filters)  which  address  the  primary  threats  believed  pertinent  to  the 
area  in  question.  These  models  range  from  large  scale  depictions  of 
possible  national  courses  of  action  for  an  adversary  country  through 
increasing  detail  as  to  the  specific  activities  and  events  that  would 
characterize  the  unfolding  of  any  of  the  major  courses  of  action, 

IWAMS  will  support  the  threat  recognition  functions  and  analysis  in  a 
number  of  ways,  including  the  following: 

•  The  threats  which  are  modelled  in  scenario  form  may 
be  stored  and  called  up. 

•  The  displayed  models  would  include  prompting  and  cueing 
devices  which  identify  for  the  analyst  the  status  of 
the  different  models,  for  example,  indicating  previous 
instances  when  certain  indicators  were  linked  to 
different  nodes  on  the  models.  Also  included  would  be 
prompting  cues  which  help  increase  the  comprehensive¬ 
ness  of  analysis,  for  example,  cues  which  aid  the 
analyst  in  conducting  a  systematic  review  of  the 
different  models  against  the  indicators  moved  forward 
from  the  monitoring  stage. 

•  The  analyst  can  change  or  modify  existing  threat 
models  on  the  basis  of  input  data?  he  will  have  the 
ability  to  generate  completely  new  models  through  a 
configurational  language  discussed  below. 
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•  The  system  will  provide  statistical  and  alphanumeric 
descriptions  of  past  changes  to  models;  show  relative 
activity  levels  as  well  as  periods  of  no  change 
associated  with  various  elements  in  models;  and 
summarize  the  changes  depicted  graphically  on  the 
displays  of  the  models  themselves.  Obviously  this 
latter  output  capability  will  provide  needed  support 
for  summaries  for  management. 

•  The  above  capabilities  will  provide  the  basis  for 
graphic  indications  of  the  ramifications  of  signifi¬ 
cation  throughout  the  series  of  models.  It  was  noted 
in  Section  4  that  when  the  analyst  changes  given 
elements  or  nodes  in  a  model,  often  the  change  reunifies 
through  the  logic  structure  of  other  related  models  and 
carries  the  implication  that  changes  must  be  made  there 
as  well  in  order  to  complete  the  full  meaning  that  has 
resulted  (and  which  ultimately  brings  changes  in 
signification  values).  IWAMS  will  at  the  least  prompt 
the  analyst  to  proceed  through  a  series  of  review 
steps  in  which  he  examines  other  structures  that  might 
be  affected  by  a  change  in  judgment  at  a  given  point. 

By  providing  such  capabilities,  IWAMS  clearly  affords  a 
number  of  significant  advantages  to  both  analysts  and  manager.  The 
IWAMS  design  reflects  the  concern  that  only  measurable  activities  of 
analysis  be  stored,  recorded  and  displayed.  The  analytic  structure 
of  models,  maps,  etc.  represents  a  coherent  system  of  analytic 
thinking,  creating  a  common  thread  throughout  the  system  in  terms  of 
a  visual  and  alphanumeric  language  to  be  used  by  analysts  and 
management . 
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Projection.  The  role  cf  IWAMS  in  the  projection  stage  is 
essentially  the  same  as  for  the  threat  recognition  stage.  The 
difference,  of  course,  is  that  IWAMS  is  now  providing  the  same 
computer-based  support  for  the  projection  maps,  matrices  and  other 
structures  used  in  projection  as  it  provided  for  the  threat  recogni¬ 
tion  models.  Thus,  IWAMS  provides  in  the  projection  stage  all  of 
the  capabilities  of  storage,  modification,  display,  and  recording, 
with  respect  to  these  elements  of  projection: 

•  Statements  of  predictions  and  forecasts  in  the  format 
of  the  basic  information  categories. 

•  Display  of  the  specificity  rating  scale,  using  color 
and  other  techniques  to  emphasize  levels. 

•  All  appropriate  computer  operations  on  sectors  and 
their  comparisons. 

•  Generation,  modification  and  display  of  single  sector 
and  probable  future  maps. 

•  Generation,  modification  and  display  of  divergence 
maps . 

•  Generation,  display  and  modification  to  rationale 
matrices  and  vector  formats. 

All  of  these  outputs  of  the  projection  stage  will  be 
reviewable  in  terms  of  past  status  versus  present  status.  Analysts 
and  managers  will  have  access  to  summaries,  using  the  system  of 
displays,  with  changes  indicated.  In  addition,  the  display  of 
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ramifications  of  meaning  throughout  the  different  structures  will 
be  provided  in  the  same  way  as  for  the  threat  models. 

Measures 


A  detailed  description  of  the  development  of  the  measures  — 
effectiveness,  readiness  and  maintenance  —  together  with  the  key 
variables,  was  provided  in  Section  3.  In  addition,  much  of  the 
discussion  in  Section  4  on  warning  analysis  methods  direccly  or 
indirectly  concerns  the  generation  of  measures  data  as  an  organic 
outcome  of  the  process  of  analysis  itself,  presuming  that  such 
analysis  involves  significant  man-machine  interactions  which  them¬ 
selves  indicate  the  occurrence,  duration  and  otner  aspects  of 
particular  steps  in  the  analysis  process.  The  key  information- 
related  variables  to  be  monitored  in  the  analytic  process  are  backlog, 

* 

relationality,  and  signification.  The  description  of  the  informal 
test  of  IWAMS  indicated  the  basic  mechanics  by  which  the  data  to 
provide  the  monitoring  of  these  variables  (and  the  subsequent 
j  derivation  of  measures)  would  be  developed.  In  Section  3,  the 
^  calculations  for  converting  the  variables  data  into  statements  of 
,  measures  were  described.  Against  this  background,  we  may  examine 
5  the  computer-based  operations  within  IWAMS  required  for  monitoring 
the  variables  and  generating  the  measures. 

Backlog.  Backlog  is  monitored  for  each  analytical  stage  and 
each  step  and  procedure  within  stages.  To  measure  backlog,  IWAMS  must 
track  and  monitor  a  number  of  elements,  including  the  following: 
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Stage 

Function 

Monitoring 

Date/time  of  arrival  of  data  items 

Time  analyst  references  data  items 

Time  indicators  generated  or  identified  and 
decla'-'-d  by  analyst. 

Threat  Recognition 

limes  in  which  indicators  are  related  to 
applicable  threat  scenarios  (i.e.,  indi¬ 
vidual  nodes  and  elements) 

Projection 

Times  that  the  output  data  from  the  threat 
recognition  stage  are  reviewed  against  the 
projection  structures.  For  example,  the 
times  in  which  the  results  of  the  threat 
recognition  analysis  are  reviewed  for  their 
impact  on  sectors,  the  various  projection 
maps ,  etc . 

The  times  in  which  thesa  activities  are 
completed . 

Rather  than  list  them  again  here  in  detail,  it  suffices  to 
point  out  that  all  the  steps  and  procedures  in  the  various  stages  of 
analysis,  as  implemented  through  the  methods  and  techniques,  must  be 
recorded  in  IWAMS  with  respect  cc-  those  times  during  which  they  were 
involved  in  the  analysis  of  data.. 


Relationality.  Relationality  will  be  measured  at  each  point 
within  each  major  stage  of  the  analysis  process.  It  was  shown  above 
that  the  model  of  warning  analysis  encompasses  a  number  of  steos  and 
procedures  in  each  of  the  major  stages.  Moreover,  a  series  of  methods 
and  techniques  of  analysis  have  been  defined  whose  enactment  results 
in  the  analysts  carrying  out  taose  seeps  and  procedures.  The  IWAMS 
approach  is  to  develop  recordable  man-machine  interactions  at  critical 
points  of  activity  in  these  sequences.  The  analyst  logs  on  and  off 
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the  IWAMS  system  and  performs  certain  analytic  steps  and  procedures 
manifest,  i  by  the  calling  up  of  displays  and  other  activities.  By 
recording  these  man-machine  activities  and  timing  them,  the  relation- 
ality  measure  is  determined. 

The  activities  and  items  monitored  at  each  stage  to  determine 
relational ity  include  the  following: 


Stage 

Monitored  Activity 

Monitoring 

Calling  up  of  input  data  for  review. 

Categorizing  of  input  data. 

Calling  up  of  predetermined  indicator  lists. 

Identification  and  display  of  correlations. 

Threat  Recognition 

Calling  up  and  displaying  of  preestablished 
threat  models  (PAMNACS,  DENS,  etc.),. 

Calling  up  of  displays  comparing  threat 
models  to  which  input  data  has  correlated. 

Projection 

Calling  up  of  threat  predictive  formats. 

Calling  up  of  specificity  rating  scale. 

Calling  up  of  sector  lists. 

Calling  up  of  probable  future  maps. 

1 


Signification.  Signification  is  also  measured  at  each 
pcint  of  analysis.  We  have  discussed  in  detail  the  strategy  for 
measuring  signification  within  IWAMS.  The  basic  mechanism  is  to 
monitor  and  record  changes  made  to  the  models  and  other  structures 
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(for  example,  projection  maps)  as  analysts  assign  meaning  to  input 
data.  Examples  of  the  items  to  be  monitored  are  the  following: 


Stage 


Monitored  Activity 


Mor storing 


Changes  in  the  activity  levels  assigned  to 
indicators . 


Changes  in  the  priorities  or  weightings 
given  indicators. 


Indicators  not  called  up  for  review  against 
given  input  data. 


Threat  Recognition 


Changes  to  the  nodes  or  other  elements  in  the 
structures  and  forms  of  analysis. 


Association  of  given  indicators  with  specific 
nodes  of  threat  models. 


Projection 


Changes  in  the  specificity  value  assigned  no 
given  projections. 

Changes  in  the  predictive  formats. 

Changes  in  the  number  of  sectors. 

Addition  of  new  sectors. 

Changes  in  possible  future  maps. 

Changes  in  divergence  maps. 

Changes  in  assumptions  matrices. 


Since  TWAMS  will  determine  the  relative  times  spent  on 
various  activities,  and  the  nature  of  those  activities,  it  will  be 
possible  to  develop  a  considerable  amount  of  insight,  based  on 
determining  such  aspects  as: 


ib 
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a  The  time  since  each  scenario,  model,  or  other  piece 
of  analytic  structure  has  been  modified. 

a  Time  spent  on  various  activities. 

a  The  dimensionality  of  given  models. 

a  Average  time  between  management  reviews  of  analytic 
status  in  various  problem  areas. 


Scenarios 


IWAMS  incorporates  scenario  language  for  use  by  analysts. 
The  term,  scenario  language,  is  used  here  to  include  all  elements 
within  the  total  structure  of  models,  projection  maps,  assumptions 
matrices,  and  other  forms  within  the  analytic  context.  The  scenario 
language  which  will  be  provided  in  IWAMS  for  the  analyst  may  be 
functionally  described  as  follows t 

•  Block  and  node  oriented. 

•  Parallel  acriviti es/sequential  actions. 

©  Scenario  building  blocks  provided. 

•  Scenario  editing  available  —  addition,  deletion  and 

modification  provided  to  analysts. 

e  Scenario  display  via  graphics,  with  color  and  dynamic 
prompting. 
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Figure  22  is  an  example  display  format  used  by  an  analyst 
working  on  the  development  or  change  of  a  given  model.  In  its  full 
design  detail,  there  would  be  a  similar  display  format  for  working  up 
any  of  the  structures,  whether  in  the  monitoring,  threat  recognition, 
or  projection  stages.  The  example  shown  in  Figure  22  is  intended  to 
be  generic  and  representative.  But  it  can  be  seen  that  an  analyst 
could  change  the  nodes  in  a  model;  add  or  subtract  structure;  and  have 
indications  of  activity  points  and  degrees  of  specificity  within  the 
models  as  indicated  by  color  and  other  techniques. 


Management  Operations 

The  last  major  function  within  the  IWAMS  system  architec¬ 
ture  is  the  management  operations  function.  IWAMS  provides  for 
multiple  levels  of  management.  At  each  level,  provision  is  made  for 
managers  to  provide  input  to  the  analyst  on  the  IWAMS  system  and  to 
receive  displayed  output  from  that  system.  Probably  a  valid  assump¬ 
tion  is  that  most,  of  the  time  higher  levels  of  management  will  have 
less  concern  for  the  day-to-day  details  of  analytic  ctivity  than 
will  the  analysts  and  their  immediate  managers.  Coii  iunications  within 
the  IWAMS  structure  between  analysts  and  management  levels  is  indeed 
an  important  design  consideration.  As  presently  configured,  IWAMS 
provides  for  linear  bi-directional  communication  as  follows:  from 
individual  analysts  to  or  from  level  1  management  (immediate  super¬ 
visor);  and  from  level  1  to  and  from  level  2  management,  and  so  on 
as  appropriate.  The  table  below  shows  simplified  examples  of  the 
inter-level  communication  types.  Also  shown  are  displays  that  would 
be  available  to  the  individual  levels. 
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IWAMS  Communication  and  Displays  by  Management  Level 


Management 

Level 

Input  Options 

Displays 

Level  1 

To: 

Analyst 

Description: 

Query  on  signication/ 
rationale 

Active  scenarios,  threats, 
projections,  etc.  by  "Desk"  and 
activity 

Query  on  scenario 
construction 

Measures  by  "Desk"  and 
activity 

Output/ activity 
direction 

Rationale  (assumptions,  etc.) 

Level  2 

Display  of  analytic 
perspectives  for 
review 

Level  2 


Level  1  Activity  priori cy 

Scemrio  critique/ 
approval 

Resource  direction 


Measures  by  center  or  "Desk" 
on  active  threats,  projec¬ 
tions 


Level  3 


Activity  Report 
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There  is  no  more  delicate  matter  to  take  in  hand,  nor  more 
dangerous  to  conduct,  nor  more  doubtful  in  its  success,  than 
to  set  up  as  a  leader  in  the  introduction  of  changes.  For  he 
who  innovates  will  have  for  his  enemies  all  those  who  are  well 
off  under  the  existing  order  of  things,  and  only  lukewarm 
supporters  in  those  who  might  be  better  off  under  the  new. 

This  . . .  results  . . .  from  the  incredulity  of  mankind,  who  will 
never  admit  the  merit  of  anything  new,  until  they  have  seen  it 
proved  by  the  event. 

Machiavelli 

For  those  responsible  for  facilitating  technical  innovation, 
the  problem  of  developing  strategies  of  successful  transfer  of  tech¬ 
nology  from  the  laboratory  to  the  operational  world  is  almost  always 
formidable.  This  clearly  is  the  case  with  warning  analytic  aids. 

The  innovation  manager  faces  a  specific  dilemma:  an  innovation 
should  be  proved  constructive  before  it  is  integrated  into  an  opera¬ 
tional  setting,  yet  the  most  credible  assessment  is  provided  by 
usage  in  an  operational  setting.  Therefore  it  becomes  necessary  to 
design  an  approach  which  can  be  used  and  demonstrated  in  an  environ¬ 
ment  that  is  either  within  the  operations  or  at  least  slightly 
off-line  and  parallel  with  the  operational  setting. 

Certainly  the  user  --  the  analyst  and  his  manager  --  is 
the  essential  factor  at  the  concluding  phases  of  transfer  of  innova¬ 
tions.  The  chief  reason  for  rejection  of  innovations  appears 
frequently'  to  be  inertia.  There  will  be  a  tendency  for  users  to 
block  changes  to  established  procedures.  With  respect  to  analysts, 
we  are  referring  to  a  group  now  beset  with  persistent  overload  condi¬ 
tions.  Other  obstacles  arise  from  factors  such  as  the  frequently 
high  rate  of  turnover  within  warning  and  watch  analyst  groups. 

The  authors  have  sought  to  design  in  IWAMS  a  system  which 
offsets  these  constraints  on  the  introduction  of  new  technology.  One 
way  of  reinforcing  these  points  is  to  indicate  that  the  entire  apprach 
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has  been  designed  to  commend  itself  to  analysts  and  managers.  We 
have  discussed  feasibility  with  many  people  and  have  found  especially 
valuable  the  perspective  of  Russell  J.  Smith,  formerly  Deputy  Director 
of  Intelligence  at  CIA.  Smith  has  encouraged  the  view  that  the 
analytic  routine  should  combine  a  broad  array  of  analytic  techniques 
in  order  to  offer  substantial  assurance  of  comprehensive  thoroughness, 
sophisticated  analysis,  and  safeguards  against  bias.  Analysts  should 
subject  their  information  and  judgment  to  a  series  of  rigorous  steps, 
thereby  gaining  confidence  in  obtaining  return  from  the  inputs. 
Supervisors  whose  analysts  work  faithfully  with  such  a  system  should 
feel  confident  that  few,  if  any,  avenues  have  been  left  unexplored  in 
the  effort  to  derive  significance  from  the  available  information. 
Moreover,  if  the  analysts  review  new  inputs  against  displayed  models 
of  previously  established  guidepoints,  and  then  employ  projection 
techniques,  the  system  should  cause  the  analyst  to  think  the  problem 
through  each  time,  to  test  the  meaning  of  new  information  against 
the  array  of  criteria,  and  to  come  to  some  decisive  judgment.  The 
interaction  of  previously  established  indicator  and  threat  models 
with  event  analyses  and  policy  statements  would  give  considerable 
assurance  that  the  analyst  would  be  encouraged  into  modes  of  thought 
where  meaning  may  be  found. 

Certainly  the  system  should  be  easy  and  natural  for  an 
analyst  to  use.  This  is  not  to  say  that  training  requirements  must 
be  minimal;  yet  as  vital  as  warning  operations  are,  training  require¬ 
ments  for  IWAMS  need  not  be  nearly  as  severe  as  those  for  piloting 
high  performance  aircraft  or  for  operation  of  various  weapon  systems. 
Analysts  should  find  the  warning  analysis  system  invariably  suppor¬ 
tive,  not  a  burden,  and  should  turn  to  it  enthusiastically  with  the 
expectation  that  it  will  help  them  find  meaning  that  might  otherwise 
be  missed.  Until  this  feeling  exists,  aids  will  be  used  perfunctorily, 
or  only  when  there  is  time  to  spare  —  not,  in  short,  in  truly 
critical  situations  —  and  possibly  gradually  abandoned  altogether. 
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In  addition,  the  system  should  be  fairly  broad  in  its  application. 

It  should  fit  the  various  kinds  of  intelligence  problems  the  analyst 
confronts . 


The  system  must  also  help  to  mitigate  the  information 
overload.  Based  or.  field  experience,  it  is  our  strong  judgment  that 
with  IWAMS  several  analysts  can  normally  reduce  typical  backlogs  of 
input  data  for  the  busiest  problem  areas  by  working  steadily  to  do 
so  through  only  parts  of  their  shifts.  There  will  ordinarily  be  time 
remaining  for  review  of  estimative  reports,  methodology  refinement  and 
research,  and  experimentation  with  analytic  structure  and  routine. 
IWAMS  provides  what  analysts  indicate  is  important:  a  context,  a 
coherent  framework,  in  which  to  place  data  being  analyzed.  There  is 
a  "home"  for  the  data.  As  we  have  noted,  a  high-context  system  tends 
to  ease  problems  of  information  overload. 


Moreover,  the  IWAMS  memory,  as  it  is  built  up,  represents 
an  extremely  coherent  and  succinct  reference  system  for  the  analyst. 
Experience  with  the  memory  will  help  analysts  document  and  decide 
which  kinds  of  data  are  more  and  less  useful.  This  capability 
could  prove  very  beneficial  to  judgments  of  the  value  of  information 
and  to  establishing  priorities  with  respect  to  the  attention  given 
different  types  of  data. 

In  terms  of  the  memory  capacity,  there  is  no  doubt  that 
providing  ample  storage  for  the  selective  memory  envisioned  for  IWAMS 
is  quite  feasible.  Indeed,  there  are  no  technical  obstacles  —  no 
software  or  hardware  "breakthroughs"  —  required  for  implementing  the 
system. 


Beyond  these  considerations,  feasibility  reduces  simply  to 
the  fact  that  we  have  met  analysts  willing  to  work  with  IWAMS.  IWAMS 
will  require  learning  and  impose  challenges.  It  will  mean  that  the 
analysts  who  use  it  must  be  able  to  interact  with  a  computer-based 
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support  system  and  function  in  the  electronic  visual  medium.  It 
will  mean  opening  up  analysis  to  greater  visibility.  But  it  will 
also  mean  an  extremely  challenging  and  engrossing  form  of  warning 
analysis,  one  that  confronts  some  of  the  most  important  and  adventure¬ 
some  national  problems.  It  will  extend  the  sense  of  a  community  of 
analysts,  with  the  community  linked  together  in  an  electronically- 
based  network  of  conceptualizing.  The  establishment  of  measures  of 
performance  and  conceptual  frameworks  shared  in  a  community  of 
talented  people  cannot  but  produce  in  the  future  excellent  method¬ 
ological  innovations  and  new  skills.  It  will  inevitably  show  the 
way  for  new  technology  applications. 

But  perhaps  we  can  best  sum  up  our  views  on  incentives  by 
relating  an  experience.  A  good  friend  who  happened  for  awhile  to  be 
in  char re  of  ail  analysts  at  a  large  varning  center  recently  adver¬ 
tized  in  the  local  papers  for  two  analyst  positions.  Over  400 
applications  were  received.  Two  persons  in  their  twenties,  with 
PhD's,  hard  and  soft  science  training,  and  familiarity  with  computer 
operations,  were  selected  after  much  deliberation  over  many  promising 
candidates.  Warning  analysis  is  clearly  a  field  for  especially 
talented  people  whose  chief  incentive  is  the  pursuit  of  understanding. 
There  are  ample  numbers  of  these  people.  And  certainly  what  will  help 
us  most  in  the  long  run  are  not  off-line  consultants  and  expert 
observers  but  innovative  practitioners  among  the  analysts  themselves. 

As  a  final  thought  on  the  esprit  of  such  analysts,  and  with 
only  a  few  reservations  (after  all,  he  was  "mad"),  there  is  sympathy 
with  an  exhortation  made  by  Captain  Ahab,  one  of  the  great  challengers 
of  the  world's  perversity: 

All  visible  objects,  man,  are  but  as  pasteboard  masks.  But  in 
each  event  --  in  the  living  act,  the  undoubted  deed  --  there, 
some  unknown  but  still  reasoning  thing  puts  forth  the  mouldings 
of  its  features  from  behind  the  unreasoning  mask*  If  man  will 
strike,  strike  through  the  mask! 
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Watershed  literature  of  warning:  Roberta  Wholstetter, 

Pearl  Harbor:  Warning  and  Decision,  Stanford,  California: 
Stanford  University  Press,  1962.  Barton  Whaley,  Codeword 
Barbarossa,  Cambridge,  Mass:  MIT  Press,  1973.  Avi  Schlaim, 
"Failures  in  National  Intelligence  Estimates:  The  Case  of 
the  Yom  Kippur  War,"  World  Politics  28  (April),  pp. 348-380. 

A  set  of  writings  recently  by  Thomas  Belden:  Thomas  G. 
Belden,  "Indications,  Warning,  and  Crisis  Operations," 
International  Studies  Quarterly,  Vol.  21  No.  1,  March  1977, 
pp.  181-198. 

In  particular,  Richards  J.  Heuer,  Jr:  Richards  J. 

Heuer,  Jr.,  "Cognitive  Biases:  Problems  in  Hindsight 
Analysis,"  Intelligence  Studies  XXI 1/2,  Summer  1978, 

pp.  21-28. 

"The  brain  itself:"  Edward  Rothstein,  "The  Dream  of  Mind 
and  Machine,"  The  New  York  Review  of  Books,  December  6,  1979, 
pp.  34-39.  A  superb  review  of  Douglas  Hofstader's  Godel , 
Escher,  Bach:  An  Eternal  Golden  Braid,  New  York:  Basic 
Books,  1979. 


2.  WARNING  ANALYSIS  MODEL 


various  organizations  and  individuals:  definitions  appear 
in  sources  such  as  Defense  Intelligence  School  textbooks. 
Joint  Chiefs  of  Staff  (JCS)  publications,  Instructions  at 
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warning  centers,  and  writings  by  individual  authors  such  as 
Cynthia  Grabo  and  Evan  Davis.  Unfortunately,  these  can  be 
difficult  to  obtain.  We  are  indebted  to  Captain  Frederick 
W.  Robitschek,  Jr.,  USAF,  for  providing  an  informal 
compendium  of  some  of  these  definitions  assembled  for  his 
research.  Some  definitions  simply  do  not  exist  in  acces¬ 
sible  documents.  The  itemized  entries  on  pages  15  and  16 
include  elements  of  several  such  definitions. 

(16)  pointing  out  that  warning  is  not:  Captain  Frederick  W. 

Robitschek,  Jr.,  USAF,  in  a  compendium  of  informal  working 
papers  generated  in  the  Pacific  Command,  1978-79.  We  are 
in  the  debt  of  Captain  Robitschek,  an  expert  on  warning 
operations,  for  a  number  of  useful  discussions  and  research 
materials . 

(21)  We  distinguish  four  types  of  projections:  these  distinc¬ 
tions  are  not  new,  though  the  terminology  and  rationale  are. 
The  difference  between  unimpeded  and  influenced  projections 
bears  on  theoretical  aspects  of  the  nature  of  international 
crises,  for  example,  the  concept  that  such  crises  and  their 
management  involve  significant  —  perhaps  decisive  — 
interactions  between  and  among  parties.  (Se  Lden, 

op  cit.  and  Charles  McClelland,  "Anticipation  of  Crises," 
International  Studies  Quarterly,  Vol.  21,  No.  1,  March  1977, 
pp.  15-38,  for  representative  discussions. 

(22)  For  each  of  the  four  types  of  projection:  we  are  indebted 
to  Thomas  Belden  for  insightful  discussions  pertaining  to 
the  uses  of  the  information  categories  for  formatting 
warning  reportage  and,  more  basically,  for  warning  analysis 
itself.  See  also  Belden,  ibid. 


SOURCE  NOTES 


3.  WARNING  ANALYSIS  MEASURES 

(30)  In  analysis  essentially  we  associate  different  data  and 

assign  meaning:  we  will  have  much  more  to  say  about  the 
dynamics  of  warning  analysis  in  Section  4.  There  we 
explore  the  interpretive  impulse  and  its  dynamics  in  some 
detail . 

(32)  Portions  of  James  Grier  Miller's  work:  see  James  Grier 

Miller,  Living  Systems.  New  York:  McGraw-Hill  Book 
Company,  1970.  We  are  indebted  to  James  Miller  for 
discussing  with  us  portions  of  his  work  and  their  utility 
in  the  present  research. 

(34)  "Meaning  represents":  Miller,  Living  Systems,  p.  11. 

(38)  Skeptics  of  post  mortems :  See  Heuer,  op.  cit. 

(39)  These  include  the  assumption  that  usually  multidimension¬ 
ality  is  good  in  warning  analysis:  this  well-founded 
assumption  is  pervasive  in  the  warning  community. 

(39)  Another  major  assumption  (concerns)  safeguard  procedures: 

this  too  is  pervasive  in  the  warning  community.  A  fine 
discussion  of  the  safeguards  issue  may  be  found  in 
Avi  Shlaim,  op.  cit. 

Chan's  work  on  warning  failures:  see  Steve  Chan's  excellent 
article,  "The  Intelligence  of  Stupidity:  Understanding 
Failures  in  Strategic  Warning."  The  American  Political 
Science  Review,  Vol.  73,  1979,  py.  171-180. 


(40) 
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(40)  "We  tell  ourselves":  See  Joan  Didion,  The  White  Album. 

New  York:  Simon  and  Schuster,  1979,  p.  11. 

(41)  "Even  though  human  behavior":  See  Edward  0.  Wilson, 

On  Human  Nature,  Cambridge,  Mass.:  Harvard  University  Press, 
1978,  p.  73. 

(42)  classification  of  the  specific  obstacles  to  analytic  effec¬ 
tiveness:  we  are  indebted  to  Shlaim,  op.  cit. ,  for  his  fine 
discussion  of  some  of  the  pitfalls  in  warning  analysis. 
Relevant  perspective  also  exists  in  a  book  by  the  Soviet 
authors,  V.  Druzhinin  and  D.  Kontorov,  Concept,  Algorithm, 
and  Decision,  translated  end  published  under  the  auspices 

of  the  United  States  Air  Force,  U.S.  Government  Printing 
Office,  Stock  #  0870-00340. 

(42)  In  Soviet  Literature:  see  Druzhinin  and  Kontorov,  Ibid. 

(43)  As  Shlaim  im  particular  discusses:  see  Shlaim,  op.  cit. 

(45)  Belden,  op.  cit. ,  p.  191. 

(45)  learning  from  "ambiguous"  and  "incomplete"  information: 

a  concept  discussed  broadly  by  Druzhinin  and  Kontorov, 
op.  cit. 


(46) 


Procedures  and  techniques  for  weighing  opposing  hypotheses: 
several  writers  on  warning  discuss  briefly  this  fundamental 
strategy  of  warning  analysis.  See  Wholstetter  and  Shlaim. 
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(46)  Procedures  that  foster  systematic  use  of  a  mixture  of 

analytic  approaches:  Willis  W.  Harman  has  written  valuably 
on  this  analytic  strategy  in  the  context  of  forecasting. 

See  Willis  W.  Harman,  An  Incomplete  Guide  to  the  Future, 

New  York:  W.W.  Norton  and  Company,  1979,  pp.  9-19. 

(51)  To  derive  real  and  estimated  effectiveness:  the  overall 

discussion  of  information-related  variables  draws  in  a  very 
general  sense  from  some  of  Miller's  conceptualizing  in 
Living  Systems,  especially  those  sections  dealing  with 
information  subsystems  in  the  individual  and  the  group. 

See  particularly  Chapters  8  and  9. 

(53)  The  relationality  variable:  insofar  as  we  know,  this 
variable  is  unique  to  the  present  conceptualization  of  the 
warning  analysis  process.  That  is,  it  is  a  new  concept  with 
respect  to  measurement  of  warning  analysis.  Of  course,  the 
notion  of  organized  procedure  is  hardly  new. 

(54)  The  most  important  of  the  three  variables,  signification: 
insofar  as  we  know,  this  variable  also  represents  a  new 
conceptualization  with  regard  to  warning  analysis .  The 
persisting  problem  of  meaning,  of  course,  is  essentially 
bound-up  with  the  signification  variable. 

(54)  Signification  is  captured  by:  There  is  awareness  of  the 

long-standing  quarrels  among  theorists  of  interpretation 
of  "narrative":  what  is  it  we  do,  and  what  is  it  we  can 
learn,  when  we  attempt  to  interpret  records  of  previous 
outlooks?  Indeed,  there  are  those  who  argue  that  recovery 
of  an  observer's  (say  a  writer's)  original  "intentions"  is 
impossible.  In  the  present  context,  as  the  ensuing 
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discussion  will  show,  the  signification  measure  involves 
this  theoretical  aspect.  However,  we  shall  postpone 
discussion  of  it  until  we  have  concluded  a  portion  of 
the  discussion  in  Section  4  below  (Methods  of  Warning 
Analysis)  for  the  signification  measure  requires  also  a 
system  of  signification  which  would  foster  interpretation 
of  previous  meaning. 

The  Soviets  distinguish:  see  Druzhnin  and  Kontorov,  op  cit. 

As  Richards  J.  Heuer,  Jr.,  has  written:  The  discussion  of 
hindsight  problems  draws  on  Heuer,  op  cit.  Sources  Heuer 
draws  on  include:  Fischhoff,  Baruch  and  Beyth,  Ruth.  "I 
Knew  It  Would  Happen:  Remembered  Probabilities  of 
Once-Future  Things,"  Organizational  Behavior  and  Human 
Performance,  11  (1975);  Fischhoff,  Baruch,  The  Perceived 
Informativeness  of  Factual  Information,  Technical  Report 
DDI-1,  Oregon  Research  Institute,  Eugene,  Oregon,  1976; 
Fischhoff,  Baruch,  "Hindsight  ?  Foresight:  The  Effect  of 
Outcome  Knowledge  on  Judgment  Under  Uncertainty,"  Journal 
of  Experimental  Psychology:  Human  Perception  and 
Performance,  I,  3  (1975). 

Carl  Sagan  (has)  stressed:  see  Carl  Sagan,  The  Dragons  of 
Eden.  New  York:  Random  House,  Inc.  1977.  See  especially 
Chapter  Eight. 

appointed  committees  who  explore  an  intelligence  failure: 

A  review  of  some  post  mortems  conducted  in  relation  to  the 
U.S.  Intelligence  Community  is  to  be  found  in  Richard  W, 
Shryock,  "The  Intelligence  Community  Post-Mortem  Program, 
1973-1975,"  Intelligence  Studies  XXI/2,  Summer,  pp.  15-28. 
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"a  configuration  within  the  brain":  see  Wilson,  op.  cit. , 
pp.  75-76. 

We  can  now  estimate:  see  Sagan,  op.  cit.,  especially 
Chapter  2. 

SIGINT:  signals  intelligence  (SIGINT);  communications 
intelligence  (COMINT);  imagery  intelligence  (IMINT);  human- 
source  intelligence  (HUMIMT). 

pathological  states  in  information  systems:  see  Miller, 
op.  cit. ,  pp.  81-82  for  a  general  discussion;  see  also 
pp.  473-480  for  a  discussion  bearing  on  the  individual 
human;  and  pp.  581-592  for  consideration  of  these  states 
in  groups.  See  also  pp.  121-202  for  an  extended  consider¬ 
ation  of  the  problem  of  information  input  overload, 
including  a  survey  of  related  research,  a  study  of  the 
pathological  effects  of  information  input  overload,  and  the 
results  of  experiments  conducted  by  Miller  and  associates. 

there  is  a  set  of  hypotheses  about  the  likely  behavior  of 
information-related  systems  under  varying  conditions:  see 
Miller,  Ibid. ,  for  an  extremely  detailed  treatement  of  such 
hypotheses,  related  to  his  structure  of  information¬ 
processing  subsystems  in  living  systems.  The  individual 
discussions  appear  at  many  points  in  Living  Systems. 

"What  happens  appear  to  be":  Didion,  op.  cit. ,  p.  44. 


"I  was  meant  to  know  the  plot":  Didion,  Ibid. , 


p.  13 . 
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4.  WARNING  ANALYSIS  METHODS 

(83)  The  physics  of  Newton:  Mas'ud  Zavarzadeh,  in  a  superb 

study  of  newer  forms  of  American  literature,  has  discussed 
the  impact  of  recent  and  current  physics,  as  well  as 
modern  communications  media,  on  the  novelist.  See  Mas’ud 
Zavarzadeh,  The  Mythopoeic  Reality,  Urbana,  Illinois: 
University  of  Illinois  Press,  1976,  pp.  3~49.  The  general 
epistemological  impact  of  both  physics  ai.d  the  media  has 
been  considered  by  a  number  of  observers,  but  Zavarzadeh 
addresses  its  specific  effects  on  contemporary  American 
literature  illuminatingly.  The  discussion  of  these 
subjects  in  the  present  study  (see  pp.  83-88)  unashamedly 
has  drawn  for  some  of  its  points  from  the  fine  study  by 
Zavarzadeh. 


"The  behavior  of  the  particle":  see  Cecil  Schneer,  The 
Evolution  of  Physical  Science.  New  York:  Grove  Press, 
1960,  p.  364. 

"(Ceorge  Smiley)  hated  the  Press":  John  LeCarre,  The 
LeCarre  Omnibus:  Call  for  the  Dead  and  a  Murder  of  Quality 
London  Victor  Gollancz  Ltd.,  1967,  p.  126. 


"The  now  communication  technologies":  Zavarzadeh,  op  cit. , 
p.  7. 


"the  official  level  of  reality":  see  Cutting  Edges :  Young 
American  Fiction  for  the  70 's,  ed.  Jack  Hicks.  New  York: 
Holt,  Rinehart,  and  Winston,  1973,  p.  537. 


(85) 
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(86)  reality  ...  is  "neither  significant  nor  absurd.":  see 

Alain  Robbe-Gillet,  For  a  New  Novel.  New  York:  Grove  Press, 
1965,-  p.  19.  Cited  by  Zavarzadeh. 


(87)  In  a  recent  study:  Zavarzadeh,  cp  cit. 

(87)  "Mistrust  and  even  fear":  Zavarzadeh,  op  cit. ,  pp.  7-8. 

(88)  "...  a  form  of  organized  chaos":  Zavarzadeh,  Ibid. ,  p.  9. 

(93)  "The  Early  Warning  and  Monitoring  System":  see  Judith 

Ayres  Daly,  Command,  Control,  and  Intelligence:  R&D  for 
Decision  and  Forecasting  Systems.  Arlington,  Virginia: 
Cybernetics  Technology  Office,  Defense  Advanced  Research 
Projects  Agency,  1979,  p.  12.  For  related  and  expanded 
descriptions,  see  the  following:  S.J.  Andriole,  Progress 
Report  on  the  Development  of  an  Integrated  Crisis  Warning 
System,  Decisions  and  Designs,  Inc.,  McLean,  Va., 

December  1976;  J.A.  Daly  and  T.R.  Davies,  The  Early  Warning 
and  Monitoring  System:  A  Progress  Report,  Decisions  and 
Designs,  Inc.,  McLean,  Va.,  July  1978. 


(94)  "Man  is  the  model-making  organism  par  excellence" :  see 
Edward  T.  Hall,  Beyond  Culture.  Garden  City,  New  York: 
Anchor  Press/Doubleday,  1976,  p  10. 

(95)  Situations  are  considered  as  "things":  V.  Druzhinin  and 
D.  Kontorov,  op  cit. ,  p.  103,  agree  that  situations  are  of 
primary  importance  to  the  intelligence  analyst.  See  also 
Hall,  op  cit. ,  pp.  113-123. 
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(95)  We  now  turn  to  specific  examples  of  analytic  forms:  Don 

R.  Harris  and  Frances  M.  Calloway,  two  colleagues,  have 
contributed  significantly  to  the  design  of  some  of  the 
analytic  forms  described  and  discussed  in  the  present 
study.  Harris,  formerly  a  CIA  analyst,  has  been  involved 
primarily  with  forms  used  in  the  threat  recognition  stage; 
Calloway  with  forms  used  in  the  projection  stage. 


(108)  The  warning  analyst  undertakes  to  develop  scenarios:  see 
Albert  Clarkson,  "Writing  and  Editing  Scenarios,"  Journal 
of  Technical  Communications,  Volume  16,  Number  2,  Second 
Quarter,  1969,  pp.  20-21.  Clarkson’s  article  is  drawn  upon 
for  the  present  discussion  on  scenarios  on  pp.  108-110. 

(111)  The  anthropologist,  Edward  Hall,  has  written  importantly 
about  the  relation  of  context  to  meaning:  see  Hall,  op  cit. 
especially  Chapters  7,  8,  and  9  (pp.  91-123). 

(112)  "...  context  ...  carries  varying  proportions  of  the  meaning: 
Hall,  Ibid. ,  pp.  75-76. 

(114)  The  full  rationale  behind  the  assignment  of  meaning: 

various  theories  of  interpretation  exist  and  reflect 
profound  arguments  on  the  limitations  of  ad  hoc  under¬ 
standing  of  meaning  recorded  in  texts  and  other  forms. 

In  later  discussions  of  signification  (see  section  below 
on  the  projection  stage)  we  will  consider  the  "hermeneutics" 
of  signification  as  it  is  being  considered  in  the  present 
context . 


(118) 

(119) 

(126) 


(130) 

(144) 

(145) 
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There  is  no  intent  to  provide  a  survey  of  various  projec¬ 
tion  techniques:  for  an  excellent  examination  of  the 
literature,  see  Richard  W.  Paiver,  Crisis  Forecasting  and 
Crisis:  A  Critical  Examination  of  the  Literature.  McLean, 
Virginia:  Decisions  and  Designs,  Inc.,  1976. 

As  Thomas  Belden  has  pointed  out:  Belden,  op.  cit. 

In  his  very  useful  book  . . .  Willis  Harman:  see  Willis  W. 
Harman,  An  Incomplete  Guide  to  the  Future.  New  York: 

W.W.  Norton  and  Company,  1979.  We  are  indebted  to  Harman, 
and  shamelessly  draw  from,  his  excellent  discussion  of  the 
methods  of  futures  research,  especially  pp.  9-19.  As  indi¬ 
cated  in  the  text  of  the  present  discussion  on  pp.  126-130 
and  139-143,  portions  of  the  consideration  of  projection 
methodology  are  drawn  exclusively  from  Harman  and  rely 
extensively  on  his  perspective  on  methodology.  All  refer¬ 
ences  to  Harman  in  these  pages  refer  to  An  Incomplete  Guide 
to  the  Future  and  are  part  of  the  discussion  therein  on 
pp.  9-19. 

Probable  Futures  Mapping:  A  technique  largely  developed 
by  a  colleague,  Frances  M.  Calloway. 

Thomas  Belden  has  modeled  this  phenomenon  as  a  decision 
stairway:  Belden,  pp.  cit. 

There  are  a  number  of  approaches:  see  Parker,  pp.  cit. 

"the  elucidation,  measurement  and  analysis":  see  Gerald 
W.  Hopple,  Mapping  the  Terrain  of  Command  Psychophysiology. 


(146) 
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McLean,  Virginia:  International  Public  Policy  Research 
Corporation,  August  1978,  p.  2. 

"As  is  customary  in  social  scientific  inquiry":  Hopple, 
Ibid. ,  p.  3 . 

"The  popular  paradigms  of  modern  psychology":  see  John  W. 
Sutherland,  "Towards  a  Middle-Range  Theory  of  Societal 
Dynamics,"  forthcoming  in  Human  Relations.  The  material 
quoted  is  from  a  draft  of  the  forthcoming  article. 

"a.  The  more  precise  the  prediction":  Belden,  op.  cit. 

Table  10:  Sutherland,  Ibid. 

Figure  19:  Sutherland,  op.  cit. 

See  the  Notes  section  for  a  brief  discussion:  It  is 
interesting  to  view  the  problem  of  interpretation  with 
respect  to  post  mortems  and  the  determination  of  real 
effectiveness.  In  Section  4,  we  discussed  the  crisis 
of  interpretation  in  terms  of  the  analyst  attempting  to 
monitor  and  project  activity  in  the  international  arena. 

But  given  that  a  record  of  meaning  would  be  created  in 
IWAMS,  we  may  also  consider  inteipretation  from  the  tradi¬ 
tional  standpoint  of  textual  interpretation.  The  IWAMS 
memory  is  based  on  a  system  of  signification  that  will 
produce  a  "text"  of  historical  meaning.  There  is  a  long¬ 
standing  debate  among  theorists  of  textual  interpretation 
concerning  how  much  objective  meaning  (the  original 
intended  meaning  of  a  given  author)  is  recoverable  by  later 
interpreters.  Obstacles  to  such  interpretation  derive  in 
part  from  the  passage  of  time  which  causes  new  readers  to 
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review  the  work  through  their  own  (new)  perspectives, 
thereby  (it  is  maintained)  obscurring  the  original  meaning. 
Taken  to  its  extreme,  this  view  results  in  the  position 
that  all  such  interpretation  is  endless,  futile  (in  terms 
of  recovering  original  meaning)  but  perhaps  important  (and 
certainly  fun)  if  you  concede  that  new  generations  of 
readers  should  use  significant  literature  of  the  past  at 
least  as  a  vehicle  for  viewing  the  reality  of  their  own 
times.  While  few  would  not  concede  that  some  meaning  in 
past  records,  especially  very  old  ones,  is  probably 
irretrievably  lost,  a  set  of  formidable  observers  stress 
the  common  sense  approach  that  there  is  obviously  a  great 
deal  of  objective  meaning  readily  recoverable,  particularly 
if  the  system  of  signification  fosters  the  recovery.  We 
have,  however,  barely  touched  here  on  the  complexities  of 
this  topic.  For  extended  discussion,  see  especially:  E.D. 
Hirsch,  The  Aims  of  Interpretation  and  Validity  in 
Interpretation;  Frank  Kermode,  The  Genesis  of  Secrecy: 

On  The  Interpretation  of  Narrative;  any  number  of  works 
concerning  New  Critics  and  New  Criticism,  a  powerful  move¬ 
ment  in  the  literary  world  influential  especially  in  the 
1940's;  Hirsch' s  review  of  Kermode' s  Genesis,  "Carnal 
Knowledge,"  The  New  York  Review  of  Books,  June  14,  1979, 
pp.  18-20;  and  Gerald  Graff's  review  of  Genesis ,  "The 
Genesis  of  Secrecy:  On  the  Interpretation  of  Narrative,  by 
Frank  Kermode,"  The  New  Republic,  June  9,  1979,  pp.  27-32. 

5.  IWAMS  Functional  Design 


(183) 


Mil  Standard  7935. 1-S:  refers  to  specifications  and 
requirements  for  computer-based  systems. 


IS 
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(186)  Input  data  will  be  listed  under  categories:  see  description 
in  Section  2  of  typical  categories  of  data  incoming  to 
warning  centers. 

(189)  Moreover,  lists  of  documents  received  by  the  library 

facilities:  assures  IWAMS  access  by  library  personnel 
as  well  as  analysts  and  managers. 

(200)  level  1  management:  an  arbitrary  designation,  not  an 

official  term  for  specific  management  echelons  in  warning 
centers . 

(202)  "Desk":  term  is  sometimes  used  to  refer  to  geographic  area 
(e.g..  North  Korea)  undergoing  intelligence  watch. 

(203)  "There  is  no  more  delicate  matter":  see  NiccolS  Machiavelli, 
The  Prince.  New  York:  The  New  American  Library  of  World 
Literature,  Inc.,  1952,  p.  55. 

(204)  the  perspective  of  Russell  J.  Smith:  the  remarks  on 
acceptance  criteria  are  based  on  an  informal  memorandum 
by  Smith. 

"All  visible  objects": 


(206) 


from  Moby  Dick. 
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